-- Logs begin at Wed 2022-08-24 15:51:40 UTC, end at Wed 2022-08-24 17:05:26 UTC. -- Aug 24 16:24:00.673301 dv-cinder-731167-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 16:24:00.680608 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 16:24:00 2022] *** Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: nodename: dv-cinder-731167-2-iscsi Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: machine: x86_64 Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: clock source: unix Aug 24 16:24:00.682595 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: pcre jit disabled Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: detected number of CPU cores: 4 Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: current working directory: / Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: detected binary path: /usr/bin/uwsgi-core Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: your processes number limit is 47680 Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: your memory page size is 4096 bytes Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: detected max file descriptor number: 2048 Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: lock engine: pthread robust mutexes Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: thunder lock: enabled Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 16:24:00.683076 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 16:24:00.723942 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: Python main interpreter initialized at 0x55ed8b2cc9c0 Aug 24 16:24:00.723942 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: python threads support enabled Aug 24 16:24:00.723942 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: your server socket listen backlog is limited to 100 connections Aug 24 16:24:00.723942 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: your mercy for graceful operations on workers is 90 seconds Aug 24 16:24:00.724325 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 16:24:00.724437 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: *** Operational MODE: preforking *** Aug 24 16:24:00.724701 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: *** uWSGI is running in multiple interpreter mode *** Aug 24 16:24:00.724701 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: spawned uWSGI master process (pid: 81611) Aug 24 16:24:00.724910 dv-cinder-731167-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 16:24:00.726041 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: spawned uWSGI worker 1 (pid: 81625, cores: 1) Aug 24 16:24:00.726041 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: spawned uWSGI worker 2 (pid: 81626, cores: 1) Aug 24 16:24:00.726041 dv-cinder-731167-2-iscsi devstack@keystone.service[81611]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 16:24:05.084972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:24:05.084972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: from cryptography.utils import int_from_bytes Aug 24 16:24:05.085771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:24:05.085771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: from cryptography.utils import int_from_bytes Aug 24 16:24:05.088038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:24:05.088038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Aug 24 16:24:05.093569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 16:24:05.093569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Aug 24 16:24:05.136205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 16:24:05.136489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 16:24:05.136571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 16:24:05.136648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 16:24:05.136715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] command line args: [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 16:24:05.136780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] command line args: [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 16:24:05.136885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 16:24:05.137041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 16:24:05.137143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ================================================================================ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 16:24:05.137217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ================================================================================ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 16:24:05.137292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] admin_token = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.137455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.137565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config_dir = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.137698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.137787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_dir = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.137870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config_source = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.137943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] debug = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_source = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] debug = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.138992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_config_append = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_config_append = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.139990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_dir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_dir = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.140970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.141797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.142977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.143881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] publish_errors = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] publish_errors = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.144951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] standard_threads = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.145967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] standard_threads = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_journal = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_json = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_journal = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_stderr = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_json = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.146958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_syslog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.147035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_stderr = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.147143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_syslog = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.147220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.147370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 16:24:05.147447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.147623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.147699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.147911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.148916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.149886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.150970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.151925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.152980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.153961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.154977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.155910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.156282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.156453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.156583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.156700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.156812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.156923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.157019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.157112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.157215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.157312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.157898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.158988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.159924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.160979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.161920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.162941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.163991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.164940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.165923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.166979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.167915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.168970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.169893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.170919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.171966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172189 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.external = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.external = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.172922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.173660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.173808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.173916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.173998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.password = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.token = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.password = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.token = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.174946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.175989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.176863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.177961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.178986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.179920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.180983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.181935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.182910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.183949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.184054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.184152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.184253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.184386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.184492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.184589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.190982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.191932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.192957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.193973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.194982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.195941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.196938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197578 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.197940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.198937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.199901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200246 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.200917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.201943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.202893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.203855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.204008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.204086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.204323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.204547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.204689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.204866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.205945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.206995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.207972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.208915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.209982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.210981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.211908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.212967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.213922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.214971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.215895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.216978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.217924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218156 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.218975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.219090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.219211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.219328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.219442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.219632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.219751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.220377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.220497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.220658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.220808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.220940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.221966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.222093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.222212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.222327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 16:24:05.222451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.222451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.222668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 16:24:05.222668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 16:24:05.251478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.251702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.251919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.252334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.252587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.252805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.260256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.261117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.289251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.289473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.290035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.290266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.348862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.353819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.399751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.399887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.399974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.400067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.559700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.559923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.560882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.561054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.561261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.594784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.595607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.595705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.595908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.595998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.596962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.597190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.607853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.608122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.623736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.623859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.623967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.624923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.625050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.625133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 16:24:05.642182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.642593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:24:05.642795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.642937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.643020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:24:05.645989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:24:05.646077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 16:24:05.646330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.646624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.653957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.654920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.663032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.663140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.665949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.668979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.669088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.669946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.670579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.691356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.691560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.692844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.693057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.693141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.693342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.697792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.698026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.698111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.698312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.698393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.698595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.705259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.705537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.705670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.705956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.709676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.710018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.710113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.710242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.710388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.710519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.722460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.722595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.722700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.722777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.722850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.722939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.723016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.723089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.723162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.723162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.729322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.729611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.729958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.730239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.733474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.734122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.736793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.737092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.737475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.737783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.741300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.742010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.744439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.745167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.748179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.748179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.748179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.748491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.748567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.748713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.748798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.749000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.749086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.749287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.749567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.749856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.764921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.765205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.765475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.765750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.766508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.767430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.767750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.768017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.773241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.774748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.779805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.779926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.780159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.780574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.780574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.791175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.791459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.791739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.792018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.792296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.792597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.792680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.792874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.792958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.793209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.793489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.793772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.794050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.794328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.808312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.809725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.811345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.812772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.815003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.816290 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.816456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.817749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.817900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.818190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.818478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.819233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.819525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.819818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.824896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.826307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.826422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.827766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.829840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.830139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.830424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.830706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.831277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.831569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.831857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.832146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.838607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.838891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.839170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.840083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.840398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.840685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.846196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.846471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.846752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.847027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.847310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.847561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 16:24:05.847645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.847846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.847939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.848949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.849832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.850032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.850309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.850664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.850851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.851110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 16:24:05.883707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ed8b2cc9c0 pid: 81625 (default app) Aug 24 16:24:05.885355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ed8b2cc9c0 pid: 81626 (default app) Aug 24 16:24:05.887209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0f5e5-85b7-4e16-b6d5-cf306fc94e7d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:05.887489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0f5e5-85b7-4e16-b6d5-cf306fc94e7d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:05.887756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0f5e5-85b7-4e16-b6d5-cf306fc94e7d None None] PATH_INFO: `/v3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:05.892073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Wed Aug 24 16:24:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:13.364408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529020-979a-488c-8be6-04802dc418ab None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:13.364919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529020-979a-488c-8be6-04802dc418ab None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:13.365005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529020-979a-488c-8be6-04802dc418ab None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:13.369251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:13.375585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:13.376573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:13.376763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:13.446956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: INFO dbcounter [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] Registered counter for database keystone Aug 24 16:24:13.466248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG oslo_db.sqlalchemy.engines [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 16:24:13.473093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writer thread running {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 16:24:13.652622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81625) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 16:24:13.652961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81625) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 16:24:13.709047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:13.710641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:13.839101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 16:24:14.279671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529020-979a-488c-8be6-04802dc418ab None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:14.348141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: INFO dbcounter [None req-bd529020-979a-488c-8be6-04802dc418ab None None] Registered counter for database keystone Aug 24 16:24:14.369480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG oslo_db.sqlalchemy.engines [None req-bd529020-979a-488c-8be6-04802dc418ab None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81626) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 16:24:14.376146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writer thread running {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 16:24:14.497098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529020-979a-488c-8be6-04802dc418ab None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:14.501646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:14.502261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:14.502539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:14.502807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:14.510929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:14.562981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING keystone.common.rbac_enforcer.enforcer [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 16:24:14.563384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0b56-2325-4b7d-a426-8e5ebba60913 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:14.573022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Aug 24 16:24:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:24:14.579635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:14.594023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42dec3-5512-4f18-a1fb-68c34632aa52 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:14.599217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:14.600323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:14.600630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:14.600901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] PATH_INFO: `/v3/projects/86cc49d474614954959fd1fd2c69fbe7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:14.602438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] RBAC: Authorizing `identity:get_project(project_id=86cc49d474614954959fd1fd2c69fbe7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:14.661273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING keystone.common.rbac_enforcer.enforcer [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 16:24:14.661609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868edbcc-425c-4b57-925a-f73b8372f538 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:14.666306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Aug 24 16:24:14 2022] GET /identity/v3/projects/86cc49d474614954959fd1fd2c69fbe7 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:16.591295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb582-8bdd-4e62-b129-925bb67fa39c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:16.591847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb582-8bdd-4e62-b129-925bb67fa39c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:16.591847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bb582-8bdd-4e62-b129-925bb67fa39c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:16.596627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:16.602144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:16.602536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:16.602689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:16.623680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:16.625265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:16.633305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 16:24:16.905925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb582-8bdd-4e62-b129-925bb67fa39c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:16.919026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bb582-8bdd-4e62-b129-925bb67fa39c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:16.923472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ea28d-81be-40f4-a18b-23f05079cd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:16.924066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ea28d-81be-40f4-a18b-23f05079cd0c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:16.924357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ea28d-81be-40f4-a18b-23f05079cd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:16.924650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ea28d-81be-40f4-a18b-23f05079cd0c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:16.932409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ea28d-81be-40f4-a18b-23f05079cd0c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:16.934202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ea28d-81be-40f4-a18b-23f05079cd0c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:16.943120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Aug 24 16:24:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:24:16.948473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:16.958192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789de0-30c2-4f42-868b-2992f393ee1a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:16.962575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9575f9c-6fdc-406c-9f01-d13c710de3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:16.963198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575f9c-6fdc-406c-9f01-d13c710de3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:16.963477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575f9c-6fdc-406c-9f01-d13c710de3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:16.963750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575f9c-6fdc-406c-9f01-d13c710de3e3 admin admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:16.965091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9575f9c-6fdc-406c-9f01-d13c710de3e3 admin admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:16.966926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9575f9c-6fdc-406c-9f01-d13c710de3e3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:16.968453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Wed Aug 24 16:24:16 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:18.974425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba56c9-a9f4-4d1f-96bc-08ec9886125c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:18.974994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba56c9-a9f4-4d1f-96bc-08ec9886125c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:18.974994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba56c9-a9f4-4d1f-96bc-08ec9886125c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:18.976180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfe429-fb12-46a4-855f-c93dfeb158fb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:18.976580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:18.976711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfe429-fb12-46a4-855f-c93dfeb158fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:18.977126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfe429-fb12-46a4-855f-c93dfeb158fb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:18.979231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:18.981264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:18.981546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:18.981820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:18.984584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:18.985053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:18.985337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.003638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:19.004885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:19.028728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG passlib.handlers.bcrypt [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81626) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 16:24:19.029201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG passlib.handlers.bcrypt [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81626) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 16:24:19.075143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:19.076797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:19.087464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 7/12] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:19.183408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 6/13] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:19.361857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:19.386276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:19.467690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ee93-cf21-46d2-83a4-728d8f1b6514 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:19.487022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:19.501283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:19.502021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:19.502303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:19.502575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.514316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:19.523446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:19.524451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:19.525283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 8/14] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:19.530297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:19.592083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33c146-c668-4d3b-bc29-ad2983bdd6bc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:19.594431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:19.612410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:19.630205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:19.630813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161671e-47e6-4993-8def-da2229b0c38c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:19.631106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:19.631393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:19.631666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.665761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:19.666377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:19.666649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:19.666913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.667574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:19.667868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:19.669296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:19.681414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 9/15] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:19.686859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:19.706742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:19.708406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:19.796707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526347b-44c9-4477-a7c4-151cc25587aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:19.815171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 7/16] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:24:19.820729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:19.823420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:19.832359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:19.833015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:19.833289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:19.835251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.835251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:19.836140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:19.837754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 10/17] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:19.843181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:19.858261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:19.928301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd50e94-535c-473c-883b-2e2d3699abb1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:19.944033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2191f-c1ba-43a0-bd01-cb3b89c606fe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:19.962757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:19.963615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:19.963898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:19.964172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.977688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:19.978297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:19.978577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:19.978851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:19.979972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:19.981507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:19.983248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 11/18] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:19.983871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:19.985786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:19.989156 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:24:20.004253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:20.005190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:24:20.006018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:24:20.006018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:24:20.006018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:20.006018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 8/19] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:20.011153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:20.072910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:20.078280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:20.097465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd82ae-8a69-4c6b-b823-2e8a4618c235 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:20.117661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6c740-796a-4fa3-b2cb-d6c712d2c86a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:20.134379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:20.134987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:20.135267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:20.135536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:20.136252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.domain.id=default, user.id=1440567f43e04ab1bb7a5b1e330db669) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:20.136582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:20.138042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571822df-39cc-4866-99ec-5502d10e80aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:20.145713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 12/20] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Aug 24 16:24:19 2022] GET /identity/v3/role_assignments?user.id=1440567f43e04ab1bb7a5b1e330db669&scope.domain.id=default&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:20.157956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:20.158684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:20.159032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:20.159373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:20.160115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:20.160532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:20.162049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373283-6e12-4393-92c2-b241882d1260 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:20.182458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 9/21] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:22.222906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67350c6-c311-4870-8134-dcf2f4346485 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.223395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67350c6-c311-4870-8134-dcf2f4346485 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.223395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67350c6-c311-4870-8134-dcf2f4346485 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:22.224917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 13/22] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:22.238732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c419cf-e086-4617-b873-b9497c965a09 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.239244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c419cf-e086-4617-b873-b9497c965a09 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.239244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c419cf-e086-4617-b873-b9497c965a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:22.265196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-31c419cf-e086-4617-b873-b9497c965a09 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:22.266440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-31c419cf-e086-4617-b873-b9497c965a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:22.332682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243c73-a8e6-4dc2-9a97-e4a136c4da38 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.333011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243c73-a8e6-4dc2-9a97-e4a136c4da38 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.333228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243c73-a8e6-4dc2-9a97-e4a136c4da38 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:22.334870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 14/23] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:22.342862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bdd5f-3997-408a-8815-f2f45180940d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.343147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bdd5f-3997-408a-8815-f2f45180940d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.343523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bdd5f-3997-408a-8815-f2f45180940d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:22.358577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 10/24] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:22.377287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-df6bdd5f-3997-408a-8815-f2f45180940d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:22.378497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-df6bdd5f-3997-408a-8815-f2f45180940d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:22.476141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 15/25] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:22.691353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9759-ae5b-45ce-87a2-62db28552a3e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.691730 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9759-ae5b-45ce-87a2-62db28552a3e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.691901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9759-ae5b-45ce-87a2-62db28552a3e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:22.693257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 11/26] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:22.698547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.698833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.699109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:22.720730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:22.721904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:22.768914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb9759-ae5b-45ce-87a2-62db28552a3e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:22.822240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 16/27] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:22.868694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:22.951038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb9759-ae5b-45ce-87a2-62db28552a3e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:22.975307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:22.988684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:22.998431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c30a66-6c3e-4f22-b3a5-69c2a1050c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:22.999076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c30a66-6c3e-4f22-b3a5-69c2a1050c72 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:22.999345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c30a66-6c3e-4f22-b3a5-69c2a1050c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:22.999606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c30a66-6c3e-4f22-b3a5-69c2a1050c72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.020173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c30a66-6c3e-4f22-b3a5-69c2a1050c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c7c30a66-6c3e-4f22-b3a5-69c2a1050c72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:23.026639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:23.027458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:23.028206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:23.032549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 12/28] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:24:23.036381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11231557-9e01-4c43-b602-aa3b3fa69cc1 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.036381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11231557-9e01-4c43-b602-aa3b3fa69cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.036381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11231557-9e01-4c43-b602-aa3b3fa69cc1 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.036381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 13/29] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:23.038743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11231557-9e01-4c43-b602-aa3b3fa69cc1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.169283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11231557-9e01-4c43-b602-aa3b3fa69cc1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:23.178989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:23.181387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6748ced8-6130-4aed-8449-cc3e98a6fb22 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:23.220815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:23.222387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.222821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.223289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.228193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:23.229168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.229790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.230058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.231111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:23.231805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:23.233430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfba58-a907-436a-8a68-d3137bf062db None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:23.242406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:23.244136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:23.249430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 14/30] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:23.255738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.256023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.256298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:24:23.263082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:23.264101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:24:23.264994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:24:23.264994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:24:23.264994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:23.266928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 17/31] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:24:23.267883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.276079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:23.277263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:23.328820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:23.351187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 15/32] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:23.356579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.377832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72248a3-b4a3-4850-962b-492142dbb206 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:23.423894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:23.425071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.425564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.426037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.426917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:23.432579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:23.509225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:23.513618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80b5ed-f93b-4627-ac8c-f7ed6ed2fd37 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:23.551139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:23.551701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.551973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.552293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.556507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:23.556507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:23.557283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 16/33] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:23 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:23.557875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.670379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 18/34] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Aug 24 16:24:23 2022] POST /identity/v3/roles => generated 219 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:23.675496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.779601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d5d6d-194b-4569-943c-774df1726e15 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:23.846143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4f8bb-7bd5-49ce-be09-b87067623469 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:23.865749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:23.882989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:23.888176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.888176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.888176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.888176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:23.888176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:23.888176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:23.888598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:23.888598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:23.888972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:23.889126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:23.892030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:23.892229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:23.894473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 19/35] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:23.895274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.914866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 17/36] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:23.916259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:23.957389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:24.030780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c5c68-8724-4289-a27c-d249d4336bd0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:24.062189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b27bf-f36e-4248-96fc-cc6519f725a3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:24.073863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:24.074487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.074764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.075090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.075772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:24.079936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:24.101299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffe1db-6169-4da1-a5b2-47f26a4fc700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:24.101897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe1db-6169-4da1-a5b2-47f26a4fc700 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.102173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe1db-6169-4da1-a5b2-47f26a4fc700 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.102443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffe1db-6169-4da1-a5b2-47f26a4fc700 None admin] PATH_INFO: `/v3/domains/default/users/1440567f43e04ab1bb7a5b1e330db669/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.104386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffe1db-6169-4da1-a5b2-47f26a4fc700 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1440567f43e04ab1bb7a5b1e330db669, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:24.105996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffe1db-6169-4da1-a5b2-47f26a4fc700 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:24.194831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 20/37] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Aug 24 16:24:23 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:24.195480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:24.256652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:24.319422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051ba6e-bd1e-43ca-b943-29f06da7acd4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:24.353950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 18/38] 10.222.0.17 () {58 vars in 1361 bytes} [Wed Aug 24 16:24:23 2022] PUT /identity/v3/domains/default/users/1440567f43e04ab1bb7a5b1e330db669/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:24.357007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05ef10-97ea-4164-8fe5-116185c40234 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.357007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05ef10-97ea-4164-8fe5-116185c40234 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.357007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05ef10-97ea-4164-8fe5-116185c40234 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.363110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 19/39] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:24.364606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6947d6-7855-4a32-ba0a-fe486a663a93 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.364903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6947d6-7855-4a32-ba0a-fe486a663a93 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.365179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6947d6-7855-4a32-ba0a-fe486a663a93 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.366408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:24.366559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 20/40] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:24.367552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.368113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.368749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.369652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:24.371528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:24.372332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab50140-e918-41d9-ad9f-722afbc96727 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.372647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab50140-e918-41d9-ad9f-722afbc96727 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.372922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab50140-e918-41d9-ad9f-722afbc96727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.393699 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fab50140-e918-41d9-ad9f-722afbc96727 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:24.395330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fab50140-e918-41d9-ad9f-722afbc96727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:24.475137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 21/41] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:24.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.490734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 21/42] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:24:24 2022] POST /identity/v3/projects => generated 301 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:24.509315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:24.510041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:24.526031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:24.596275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 22/43] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:24.755621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:24.852245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:24.858667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a34c-129d-4a23-af97-be888163eb06 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:24.862181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:24.899576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:24.911477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:24.911477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:24.911477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:24.911477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:24.911477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:24.968079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbfb00-7746-4995-a985-65657cfc6c43 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:24.989917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 22/44] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Aug 24 16:24:24 2022] POST /identity/v3/roles => generated 223 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:25.008106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc2b3a-042a-4535-b85f-0c5a7b8b8af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:25.008719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc2b3a-042a-4535-b85f-0c5a7b8b8af6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:25.009001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc2b3a-042a-4535-b85f-0c5a7b8b8af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:25.009277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc2b3a-042a-4535-b85f-0c5a7b8b8af6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:25.010506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc2b3a-042a-4535-b85f-0c5a7b8b8af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:25.012155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc2b3a-042a-4535-b85f-0c5a7b8b8af6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:25.017953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 23/45] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:25.020922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:25.142879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9393efa-7bd7-429f-930f-fe14aafee5a8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:25.179473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2191d7b-7539-42b0-9a37-617afe8bf4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:25.180092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191d7b-7539-42b0-9a37-617afe8bf4a0 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:25.180763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191d7b-7539-42b0-9a37-617afe8bf4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:25.180763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2191d7b-7539-42b0-9a37-617afe8bf4a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:25.181370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191d7b-7539-42b0-9a37-617afe8bf4a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:25.182835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2191d7b-7539-42b0-9a37-617afe8bf4a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:25.383843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 23/46] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:24:25 2022] POST /identity/v3/projects => generated 312 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:26.350714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c0c91-f66b-41c8-8f3c-11dc18298d3d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:26.351255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c0c91-f66b-41c8-8f3c-11dc18298d3d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:26.351255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c0c91-f66b-41c8-8f3c-11dc18298d3d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:26.352656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:26.358094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:26.358388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:26.358665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:26.377873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:26.379421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:26.381557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:26.448241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 24/48] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:26.715901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c0c91-f66b-41c8-8f3c-11dc18298d3d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:26.731988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:26.818999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c0c91-f66b-41c8-8f3c-11dc18298d3d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:26.852960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:26.853585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:26.853865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:26.854135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:26.864419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:26.869143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:26.869982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:26.870880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 25/49] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:26.876544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:26.975844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769fc91-3dfb-41ee-a1a1-dd88994bc92b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:27.010056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:27.010665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.010941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.011212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.011883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:27.012181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:27.013613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:27.019309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:27.022194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 25/50] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:27.027787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:27.213404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:27.238479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56744b54-1051-40dd-b891-bdbe848b4707 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:27.273116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74f1e5-8e85-4a26-8b21-91b2c96f8354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:27.273747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74f1e5-8e85-4a26-8b21-91b2c96f8354 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.274025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74f1e5-8e85-4a26-8b21-91b2c96f8354 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.274293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74f1e5-8e85-4a26-8b21-91b2c96f8354 None admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.275428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74f1e5-8e85-4a26-8b21-91b2c96f8354 None admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:27.276906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74f1e5-8e85-4a26-8b21-91b2c96f8354 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:27.278490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 26/51] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:27 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:27.284367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:27.297511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266258a9-cd56-40d7-ab05-bd8a56f349e9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.297795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266258a9-cd56-40d7-ab05-bd8a56f349e9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.298102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266258a9-cd56-40d7-ab05-bd8a56f349e9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.299443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 27/52] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:27.304330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.304642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.304922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.322791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:27.323997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:27.387522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8783dc-1c86-4e3a-a2f4-2e8de220499f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:27.391881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 28/53] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:27.421566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:27.422546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.422834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.423113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.424239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:27.425740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:27.427312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:27.433115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:27.501580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:27.530291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17b51-3df7-466a-8f65-ba770b785b81 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:27.565577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:27.566178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.566510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.566794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.567514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.domain.id=default, user.id=1440567f43e04ab1bb7a5b1e330db669) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:27.567814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:27.569279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:27.576969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 29/55] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Aug 24 16:24:27 2022] GET /identity/v3/role_assignments?user.id=1440567f43e04ab1bb7a5b1e330db669&scope.domain.id=default&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:27.660575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:27.732559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:27.768003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5eca5-a81d-4afc-b6f7-e28f6d7461fe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:27.802050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df964a5a-39af-48f7-a1a7-96bcb44556a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:27.802681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964a5a-39af-48f7-a1a7-96bcb44556a6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.802956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964a5a-39af-48f7-a1a7-96bcb44556a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.803225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df964a5a-39af-48f7-a1a7-96bcb44556a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.804314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df964a5a-39af-48f7-a1a7-96bcb44556a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:27.805826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df964a5a-39af-48f7-a1a7-96bcb44556a6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:27.807388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 27/56] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:27.813146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:27.911319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b6e09-fb79-4c5e-b77f-b7d01f69c41a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:27.956237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c1a51-813f-4fc4-ae50-7b30bfe0c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:27.956887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1a51-813f-4fc4-ae50-7b30bfe0c276 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:27.957172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1a51-813f-4fc4-ae50-7b30bfe0c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:27.957446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c1a51-813f-4fc4-ae50-7b30bfe0c276 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:27.958096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1a51-813f-4fc4-ae50-7b30bfe0c276 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:27.959559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c1a51-813f-4fc4-ae50-7b30bfe0c276 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:27.965375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:28.023863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 30/57] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Aug 24 16:24:27 2022] POST /identity/v3/projects => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:29.914863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602361d5-5208-4373-9c49-3ad321c2be9e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:29.915319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602361d5-5208-4373-9c49-3ad321c2be9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:29.915405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602361d5-5208-4373-9c49-3ad321c2be9e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:29.916789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 28/58] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:29.922196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:29.922563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:29.922775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:29.942286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:29.943789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:30.017746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 31/59] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:30.295206 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602361d5-5208-4373-9c49-3ad321c2be9e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:30.400536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602361d5-5208-4373-9c49-3ad321c2be9e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:30.448722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ad9b9-c81b-4e8e-9820-d6daf75a4bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:30.449357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ad9b9-c81b-4e8e-9820-d6daf75a4bcf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:30.449633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ad9b9-c81b-4e8e-9820-d6daf75a4bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:30.449904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ad9b9-c81b-4e8e-9820-d6daf75a4bcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:30.451026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ad9b9-c81b-4e8e-9820-d6daf75a4bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:30.452523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ad9b9-c81b-4e8e-9820-d6daf75a4bcf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:30.454917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 29/60] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:30.460434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:30.556903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a95d-0acf-4a83-b40a-b31f1f8c0592 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:30.573883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:30.589673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa71ac-da81-4e58-8db5-d11a7f0527d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:30.590271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa71ac-da81-4e58-8db5-d11a7f0527d2 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:30.590548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa71ac-da81-4e58-8db5-d11a7f0527d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:30.590819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa71ac-da81-4e58-8db5-d11a7f0527d2 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:30.591466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa71ac-da81-4e58-8db5-d11a7f0527d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:30.592946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa71ac-da81-4e58-8db5-d11a7f0527d2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:30.685698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 32/61] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:24:30 2022] POST /identity/v3/users => generated 275 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:24:33.863464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb97ac-cf9d-4e39-81be-8040d2fe6957 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:33.864592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb97ac-cf9d-4e39-81be-8040d2fe6957 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:33.865190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb97ac-cf9d-4e39-81be-8040d2fe6957 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:33.867286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:33.873328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59fc94-19e9-4f53-859c-32db2e3623c9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:33.874420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59fc94-19e9-4f53-859c-32db2e3623c9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:33.875547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59fc94-19e9-4f53-859c-32db2e3623c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:33.927336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dc59fc94-19e9-4f53-859c-32db2e3623c9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:33.929277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dc59fc94-19e9-4f53-859c-32db2e3623c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:34.051259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 33/63] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:34.154004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a8b3-33ef-4bf5-b4b4-39fc9d33ed34 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.154596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a8b3-33ef-4bf5-b4b4-39fc9d33ed34 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.155161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a8b3-33ef-4bf5-b4b4-39fc9d33ed34 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.157650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 31/64] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:34.159320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141a4e1-7746-4a7d-8386-81b385a76fdc None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.159821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141a4e1-7746-4a7d-8386-81b385a76fdc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.160305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141a4e1-7746-4a7d-8386-81b385a76fdc None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.162670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc315d-e382-43d7-b94b-1c8688c4714d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.163144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc315d-e382-43d7-b94b-1c8688c4714d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.163606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc315d-e382-43d7-b94b-1c8688c4714d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.167310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 34/65] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:34.176369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0382a-e7a7-4c3b-a09f-46db0f01b04b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.176369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0382a-e7a7-4c3b-a09f-46db0f01b04b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.176369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0382a-e7a7-4c3b-a09f-46db0f01b04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.176720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:34.190600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dccc315d-e382-43d7-b94b-1c8688c4714d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:34.191776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dccc315d-e382-43d7-b94b-1c8688c4714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:34.211978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-39e0382a-e7a7-4c3b-a09f-46db0f01b04b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:34.213208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-39e0382a-e7a7-4c3b-a09f-46db0f01b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:34.301759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 35/66] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:34.337857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 32/67] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:34.402935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034983-3a31-4c0e-8dc9-f2a0fae8a3cb None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.403579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034983-3a31-4c0e-8dc9-f2a0fae8a3cb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.404125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034983-3a31-4c0e-8dc9-f2a0fae8a3cb None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.406282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 36/68] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:34.412054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e365d7-3c77-4b78-94c0-800041d8ce6b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.412371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e365d7-3c77-4b78-94c0-800041d8ce6b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.412659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e365d7-3c77-4b78-94c0-800041d8ce6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.431556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b4e365d7-3c77-4b78-94c0-800041d8ce6b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:34.432843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b4e365d7-3c77-4b78-94c0-800041d8ce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:34.495693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062856f4-a660-4ab1-8cb8-e372f97b00e4 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.496314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062856f4-a660-4ab1-8cb8-e372f97b00e4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.496943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062856f4-a660-4ab1-8cb8-e372f97b00e4 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.499099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 37/69] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:34.500384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062856f4-a660-4ab1-8cb8-e372f97b00e4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:34.508985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 33/70] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:34.510577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.510577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.510828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.528899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:34.530257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:34.544860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:34.619288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:34.621782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062856f4-a660-4ab1-8cb8-e372f97b00e4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:34.635959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 34/71] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:34.661754 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:34.662707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.663373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.664075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.678621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:34.680488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:34.686466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:34.687899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:34.688761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 38/72] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:34.690022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:34.788832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:34.795572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4201824-0f3e-4aae-9a56-28e40928ad75 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:34.797740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b22d6a-d8d1-4498-acd7-dcb037f11f96 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:34.833514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:34.834373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.834795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.835237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.835645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:34.836582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:34.837194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:34.837664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:34.846597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:34.849233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:34.851416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:34.852238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:34.853037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 39/73] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:34.853672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:34.862328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:34.863218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:34.864471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:34.865513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 35/74] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:24:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:34.866071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:34.928196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:34.964623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f56bfd-a8f0-46d0-92bb-4eeefd94eef4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:34.976592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d19d00-9dc6-4c64-8387-8a6b2b92580b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.002901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043473f-c306-4038-ae55-df56a20318df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.003660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043473f-c306-4038-ae55-df56a20318df None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.004016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043473f-c306-4038-ae55-df56a20318df None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.004486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043473f-c306-4038-ae55-df56a20318df None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.005310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043473f-c306-4038-ae55-df56a20318df None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:35.005659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043473f-c306-4038-ae55-df56a20318df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.007108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043473f-c306-4038-ae55-df56a20318df None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.016093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.016534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 40/75] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.016733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.017033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.017203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043473f-c306-4038-ae55-df56a20318df None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.017290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.017948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:35.018359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.019779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.028111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 36/76] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.028111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.041285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.122734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043473f-c306-4038-ae55-df56a20318df None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.128329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafaaec-a51e-4a16-87ee-fa5ed28800c0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.144634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.158603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.159237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.159518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.159792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.160936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.162033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.162395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.162640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.162915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.163182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.163839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:35.168411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 41/77] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.168538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.168612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.172745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.174761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 37/78] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.179394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.254639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.270653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9665f-cedb-40c2-86de-0473711418fa None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.279972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef840d7-7940-40d3-b945-6c7717b39e91 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.303336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.303952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.304229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.304526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.314385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:35.315148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.315777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.316056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.316419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:35.319506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:35.320431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:35.321240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 42/79] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:35.321859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.331797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.333354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.337376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 38/80] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.337376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.379147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.441715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151836dd-5aa9-4641-aeeb-78efd1b8bb9d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.450877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cd426-aca6-42e9-b54a-3903041a25c9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.474805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.475434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.475711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.475983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.477141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.478601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.480172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 43/81] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.480967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.484608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.485461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.485744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.486021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.486664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.488110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.497326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.581413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491c016-2bf3-4352-89b8-72eacf89354f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.581920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 39/82] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:24:35 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:35.585354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.624049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.626256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.626572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.626854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.628204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.629747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.632540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 44/83] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.633179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.696036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.708820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0900760-f08c-4750-aa4d-5d6d371b2589 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.735762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:35.756619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a5d14-5686-4812-9474-a1cafbcdf790 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.759531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.760205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.760521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.760800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.761506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:35.762378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.764978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.775411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 40/84] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:35.776334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:35.802682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.803301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.803578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.803844 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.906424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:35.908024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:35.909864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405323c-ef89-4e54-a9e9-0a77738939aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:35.951767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:35.952320 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:35.953347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:35.953756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:35.986204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.013564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.041636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.041892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 45/85] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:36.042949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.043549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.159180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 41/86] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:36.160369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.175596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bba5-6d2d-454a-bfeb-20a7e2ad9bcf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.218996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.224095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.224777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.225159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.225447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.282256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.288041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812a1ad-b6e4-4bc5-bdba-619b8d3f00cf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.303112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.304837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.333337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.334697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.335289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.335814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.337404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.339581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.342811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 42/87] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:36.344279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.414916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 46/88] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:36.417145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.476631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6798a2-970c-4ea0-8a13-4c7039906e4f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.501093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.513772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.514565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.514952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.515421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.516689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.520458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.520569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.520569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 43/89] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:36.521183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.540797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579990d7-299c-4542-9e26-6e854c3faabf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.580224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.581364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.581364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.581714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.582932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.584612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.586495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:36.587473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.641851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74a8fb-b3c5-4e6c-85d7-2dd70b8b3cc7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.679025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.680048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.680521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.681066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.703014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438470-4758-4c80-8067-31b7f412dc59 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.742737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.743682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.743858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.744120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.745363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.746883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.748016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.751995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 48/91] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:36.756716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.763533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.766005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.778520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:36.900971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4bf5c-5c33-4b67-98a4-5ed625c409e9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:36.911351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 44/92] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:36.913482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:36.949091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:36.949823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:36.950257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:36.950594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:36.951787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=a94c4cb016bc4be3a26160a9d7f8b399) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:36.952984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:36.956202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:36.963310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 49/93] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/role_assignments?user.id=a94c4cb016bc4be3a26160a9d7f8b399&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:36.964372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:37.037879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:37.059135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791bb3-88fa-498b-a0f1-863bdf22cffb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:37.094839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:37.126218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1428ce-40aa-42f3-82d2-62fb9c9cab02 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:37.126602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:37.127691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:37.128171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:37.128813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:37.130919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=1440567f43e04ab1bb7a5b1e330db669) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:37.130919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:37.133265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:37.159624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 45/94] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/role_assignments?user.id=1440567f43e04ab1bb7a5b1e330db669&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:37.160128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:37.177362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:37.178237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:37.178661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:37.179070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:37.179998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] RBAC: Adding query filter params (role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=a94c4cb016bc4be3a26160a9d7f8b399) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:37.180508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:37.188367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:37.204451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 50/95] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:36 2022] GET /identity/v3/role_assignments?user.id=a94c4cb016bc4be3a26160a9d7f8b399&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:37.376191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c99c-cd2a-48c0-8ab8-da5de5a5adc8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:37.438241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43d3e8-e728-4b43-baa6-2eadf0013ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:37.439134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d3e8-e728-4b43-baa6-2eadf0013ede None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:37.439558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d3e8-e728-4b43-baa6-2eadf0013ede None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:37.440016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43d3e8-e728-4b43-baa6-2eadf0013ede None admin] PATH_INFO: `/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:37.441385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43d3e8-e728-4b43-baa6-2eadf0013ede None admin] RBAC: Authorizing `identity:get_project(project_id=ba2e2b3e0c9b4f37a2290294239e5555)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:37.442995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43d3e8-e728-4b43-baa6-2eadf0013ede None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:37.444968 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 46/96] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:37 2022] GET /identity/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555 => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:37.459376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:37.577103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a09769-2e12-45c6-8d56-b2bdce806223 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:37.615605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffac917-087c-43ca-baba-fa80805038be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:37.616637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac917-087c-43ca-baba-fa80805038be None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:37.617121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac917-087c-43ca-baba-fa80805038be None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:37.617635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac917-087c-43ca-baba-fa80805038be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:37.618584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffac917-087c-43ca-baba-fa80805038be None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=ba2e2b3e0c9b4f37a2290294239e5555, user.id=a94c4cb016bc4be3a26160a9d7f8b399) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:37.619104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffac917-087c-43ca-baba-fa80805038be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:37.620887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffac917-087c-43ca-baba-fa80805038be None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:37.628758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 51/97] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:37 2022] GET /identity/v3/role_assignments?user.id=a94c4cb016bc4be3a26160a9d7f8b399&scope.project.id=ba2e2b3e0c9b4f37a2290294239e5555&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:38.125326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f79a4-ea91-4ade-afd8-eb06efd5bf9e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:38.125861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f79a4-ea91-4ade-afd8-eb06efd5bf9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:38.126148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f79a4-ea91-4ade-afd8-eb06efd5bf9e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:38.127888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 47/98] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:38.133371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:38.133806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:38.134118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:38.153681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:38.154798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:38.157725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:38.206687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e4f03-95bc-45f3-9528-6ca0e09634a1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:38.207299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e4f03-95bc-45f3-9528-6ca0e09634a1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:38.207795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e4f03-95bc-45f3-9528-6ca0e09634a1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:38.209368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 48/99] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:38.222193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:38.222786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:38.223257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:38.234334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 52/100] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:38.258740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:38.260288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:38.290908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:38.362100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 49/101] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:38.688047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:38.810145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:38.827851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78d50-502d-4761-ace1-fd7e30528cd0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:38.877637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:38.878618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:38.879127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:38.879603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:38.897273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:38.902948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:38.913474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:38.923461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 53/102] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:38.924211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:39.069789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79527dd9-29f5-4af0-b74d-7494adb48b55 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:39.112546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:39.113166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:39.113430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:39.113702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:39.114374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:39.114674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:39.116059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:39.135557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 54/103] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:39.141571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:39.167839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2a5c9-4789-461a-8192-e4dd506db6f3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:39.241988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:39.242647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:39.242929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:39.243202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:39.249528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:39.251094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:39.252777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 50/104] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:39.261377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:39.270552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:39.316729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:39.356276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f5d88-1399-483b-a20c-42fa7ddbdd9c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:39.449930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:39.450570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:39.450850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:39.451119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] PATH_INFO: `/v3/users/6a2d55b1385f41fa975478f0a05cb942` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:39.479390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcce57-ef63-48d6-a5e2-8a7fab750d74 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:39.481391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] RBAC: Authorizing `identity:get_user(user_id=6a2d55b1385f41fa975478f0a05cb942)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:39.488361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:39.488361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 55/105] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:39 2022] GET /identity/v3/users/6a2d55b1385f41fa975478f0a05cb942 => generated 289 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:39.490461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:39.539793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:39.540804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:39.541766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:39.542288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:39.543158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:39.545574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:39.659235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 51/106] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:24:39 2022] POST /identity/v3/projects => generated 302 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:24:39.679147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a87f8-d3dd-4ec6-80d8-ca280a4add7e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:39.709183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:39.734471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe25351-7841-4b5b-aaea-d501005acaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:39.735101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe25351-7841-4b5b-aaea-d501005acaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:39.735379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe25351-7841-4b5b-aaea-d501005acaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:39.735649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe25351-7841-4b5b-aaea-d501005acaa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:39.820985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe25351-7841-4b5b-aaea-d501005acaa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:39.822874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe25351-7841-4b5b-aaea-d501005acaa7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:39.932994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 56/107] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:24:39.939113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:40.029469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:40.032689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a85dc2-e72a-4647-8f96-ed9276217a50 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.032974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a85dc2-e72a-4647-8f96-ed9276217a50 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.033247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a85dc2-e72a-4647-8f96-ed9276217a50 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.034592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 57/108] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:40.045765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6939097-5716-4138-971b-0b6510817afc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.050439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6939097-5716-4138-971b-0b6510817afc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.050439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6939097-5716-4138-971b-0b6510817afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.071591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a6939097-5716-4138-971b-0b6510817afc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:40.072823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a6939097-5716-4138-971b-0b6510817afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:40.089322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7423fe-4d85-4710-a01a-601e7a68aa35 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:40.103994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:40.151523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7339c8-79c3-4d76-b179-5086b92c9cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:40.152222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7339c8-79c3-4d76-b179-5086b92c9cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.152537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7339c8-79c3-4d76-b179-5086b92c9cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.152814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7339c8-79c3-4d76-b179-5086b92c9cb0 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.156375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7339c8-79c3-4d76-b179-5086b92c9cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:40.158537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7339c8-79c3-4d76-b179-5086b92c9cb0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:40.159628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 52/109] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:39 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:40.160833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d8e9-cac5-40e9-998e-1ef0cc1f532d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.161107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d8e9-cac5-40e9-998e-1ef0cc1f532d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.161383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d8e9-cac5-40e9-998e-1ef0cc1f532d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.162808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 53/110] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:40.167231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9d8e9-cac5-40e9-998e-1ef0cc1f532d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:40.196390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 58/111] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:40.197407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b9f1d-59ab-401e-92c0-45e9fd02d85a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.197691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b9f1d-59ab-401e-92c0-45e9fd02d85a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.197969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b9f1d-59ab-401e-92c0-45e9fd02d85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.231343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c28b9f1d-59ab-401e-92c0-45e9fd02d85a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:40.233508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c28b9f1d-59ab-401e-92c0-45e9fd02d85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:40.348292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9d8e9-cac5-40e9-998e-1ef0cc1f532d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:40.467019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 59/112] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:40.467188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:40.467188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.467188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.467188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.467470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] RBAC: Adding query filter params (role.id=e4bd5c0c6a9d4d1d97b5fa821356d545, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=6a2d55b1385f41fa975478f0a05cb942) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:40.467470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:40.467470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ccf56-57ef-4161-9e00-59561844fa37 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:40.467470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 54/113] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:40 2022] GET /identity/v3/role_assignments?user.id=6a2d55b1385f41fa975478f0a05cb942&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:40.492541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ee5a3-df2c-4ee0-aa96-678abf0e618f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.492858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ee5a3-df2c-4ee0-aa96-678abf0e618f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.493111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ee5a3-df2c-4ee0-aa96-678abf0e618f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.494672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 60/114] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:40.500884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.501502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.501966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.511824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:40.523196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:40.530353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:40.610317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ee5a3-df2c-4ee0-aa96-678abf0e618f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:40.624937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 55/115] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:40.704893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:40.735126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ee5a3-df2c-4ee0-aa96-678abf0e618f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:40.775913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1b1d6-f69e-4349-993b-b6f3ac048c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:40.776567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1b1d6-f69e-4349-993b-b6f3ac048c69 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.776889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1b1d6-f69e-4349-993b-b6f3ac048c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.777158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1b1d6-f69e-4349-993b-b6f3ac048c69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.797186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1b1d6-f69e-4349-993b-b6f3ac048c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-59d1b1d6-f69e-4349-993b-b6f3ac048c69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:40.802056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:40.802936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:40.803799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:40.807201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 61/116] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:40.813679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:40.837231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad223e4-5bc2-4943-995e-17d0a98410c1 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.837639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad223e4-5bc2-4943-995e-17d0a98410c1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.837913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad223e4-5bc2-4943-995e-17d0a98410c1 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.839682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 62/117] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:40.871756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:40.872085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:40.872361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:40.913019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:40.914237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:41.021111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c68fc-351a-4550-beb6-cc9abe114659 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.050555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 63/118] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:41.056239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.090230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.091253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.091740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.092212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.093919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:41.094441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:41.096051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:41.097729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:41.106062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 56/119] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:41.107116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.204168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:41.242183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3f525-0efc-4541-8928-838ff9edc14a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.286959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.287928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.289432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.289921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.311063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:41.318084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:41.318928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:41.319688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:41.321776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 64/120] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:41.322404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.363871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c71142-24b1-4556-9e9a-5407b2dfdfe4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.404768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.406897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.407391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.408234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.428585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:41.435812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:41.437470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:41.438419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:41.440374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 57/121] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:41.446279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.479855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de3dc6-a092-43e7-aedb-fd72bba150e2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.503467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:41.515918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:41.526153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.527481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.527966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.528806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.531102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:41.533789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:41.536037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 65/122] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:41.537481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.553069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21f92-241e-4bfc-a81c-1a14af436452 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.593029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.593641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.593920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.594228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.595047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:41.595351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:41.596931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:41.605667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 58/123] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:41.610429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.672110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935cf22-9f68-4f15-82a2-3b2970b6b569 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.711102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.711845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.712128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.712436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.717124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1df47b-2c8c-4637-a72d-1740176f9f64 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.726813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:41.734534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:41.735401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:41.736154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:41.737565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 66/124] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:41.743156 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:41.744238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.761274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.773296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.773296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.773296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.773296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:41.773296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:41.773296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:41.781304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 59/125] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:41.782500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.822169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:41.881212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086dd93-7bed-4b40-ac99-d597ccca5050 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.911487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acea83-dc1b-49b9-8cfe-d759ea3c87da None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:41.922027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.923215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.923579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.923955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.925554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:41.927401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:41.930018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 67/126] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:41.930957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:41.954401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:41.955448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:41.955973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:41.956462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:41.957887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:41.959708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:41.961939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 60/127] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:41.962876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.027086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.059382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21cc3-0a74-4722-80c8-38a514f62251 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.081477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8d843-bc80-4a63-aade-9899bdb02074 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.088421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.101798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.102724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.103299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.103780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.104696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:42.105230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.106999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.118619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 68/128] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:42.119953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.129054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.129054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.129054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.129054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.129362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.129362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.130223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 61/129] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:41 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:42.135469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.231712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32aa54-bcff-4280-ace2-5ff21ad02fce None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.243230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816112f1-357d-40bb-9adc-b9f5282244c4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.263014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.277392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.278057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.280084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.280084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/a94c4cb016bc4be3a26160a9d7f8b399/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.281412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] RBAC: Authorizing `identity:create_grant(project_id=e50fa06fb1774ed1b1cc4471b8e86025, user_id=a94c4cb016bc4be3a26160a9d7f8b399, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.282859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.289425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.290355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.290847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.291340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.293544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.296995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.306173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 62/130] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:42 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:42.306173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.342433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 69/131] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:42 2022] PUT /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/a94c4cb016bc4be3a26160a9d7f8b399/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:42.344134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.406476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.423570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ca18-87b4-409a-a901-2e556c359459 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.464420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c4937-38f9-4b4a-a856-d66357cab64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.465079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c4937-38f9-4b4a-a856-d66357cab64f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.465463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c4937-38f9-4b4a-a856-d66357cab64f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.465697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c4937-38f9-4b4a-a856-d66357cab64f None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.466898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c4937-38f9-4b4a-a856-d66357cab64f None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.468904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c4937-38f9-4b4a-a856-d66357cab64f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.470942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 63/132] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:42 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:42.472540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fd844-ad15-4e81-9e3c-b7d82bfee297 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.472654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fd844-ad15-4e81-9e3c-b7d82bfee297 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.472862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fd844-ad15-4e81-9e3c-b7d82bfee297 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.478593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 64/133] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:42.479445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fd844-ad15-4e81-9e3c-b7d82bfee297 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.484517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5209dfe-4563-4cdc-9e4e-7af9d86be947 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.528839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.529489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.529775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.530054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.531771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.533478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.535331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 70/134] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:42 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:42.535940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.549689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.602379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fd844-ad15-4e81-9e3c-b7d82bfee297 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.644541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.645662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.646042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.646407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/a94c4cb016bc4be3a26160a9d7f8b399/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.648753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] RBAC: Authorizing `identity:create_grant(project_id=e50fa06fb1774ed1b1cc4471b8e86025, user_id=a94c4cb016bc4be3a26160a9d7f8b399, role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.650308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.668950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bbc0d-514e-4745-b3cb-da3bd56ff03e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.683222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.703391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9fed3-de32-4c4e-a157-e2084484ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.704020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9fed3-de32-4c4e-a157-e2084484ff1b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.704304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9fed3-de32-4c4e-a157-e2084484ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.704616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9fed3-de32-4c4e-a157-e2084484ff1b None admin] PATH_INFO: `/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.705746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9fed3-de32-4c4e-a157-e2084484ff1b None admin] RBAC: Authorizing `identity:get_project(project_id=ba2e2b3e0c9b4f37a2290294239e5555)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.707200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9fed3-de32-4c4e-a157-e2084484ff1b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.708944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 71/135] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:42 2022] GET /identity/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:42.710058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4446de7-463a-4b9b-8148-761f2011ac06 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.710345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4446de7-463a-4b9b-8148-761f2011ac06 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.710623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4446de7-463a-4b9b-8148-761f2011ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.728082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e4446de7-463a-4b9b-8148-761f2011ac06 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:42.729343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e4446de7-463a-4b9b-8148-761f2011ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:42.759547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 65/136] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:42 2022] PUT /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/a94c4cb016bc4be3a26160a9d7f8b399/roles/df3ae0f0715049bf9ef8923719615f92 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:42.768568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.842199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 72/137] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:42.845063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2519-b84a-4e16-8365-6f8613a92a29 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.845768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2519-b84a-4e16-8365-6f8613a92a29 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.846057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2519-b84a-4e16-8365-6f8613a92a29 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.848482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 73/138] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:42.852060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb2519-b84a-4e16-8365-6f8613a92a29 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:42.897448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb6f88-fdac-4818-9976-ba7595294245 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:42.918741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:42.949647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357a069-4cb1-4d51-83ed-bf44ff551834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:42.950327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357a069-4cb1-4d51-83ed-bf44ff551834 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:42.950780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357a069-4cb1-4d51-83ed-bf44ff551834 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:42.951070 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357a069-4cb1-4d51-83ed-bf44ff551834 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/1440567f43e04ab1bb7a5b1e330db669/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:42.955524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357a069-4cb1-4d51-83ed-bf44ff551834 None admin] RBAC: Authorizing `identity:create_grant(project_id=e50fa06fb1774ed1b1cc4471b8e86025, user_id=1440567f43e04ab1bb7a5b1e330db669, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:42.957065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357a069-4cb1-4d51-83ed-bf44ff551834 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:42.987892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb2519-b84a-4e16-8365-6f8613a92a29 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:43.003350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:43.025990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:43.029317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:43.033970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 66/139] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:42 2022] PUT /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/1440567f43e04ab1bb7a5b1e330db669/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:43.035207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:43.035497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:43.036205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:43.036470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:43.036829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] PATH_INFO: `/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555/users/a94c4cb016bc4be3a26160a9d7f8b399/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:43.040594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba2e2b3e0c9b4f37a2290294239e5555, user_id=a94c4cb016bc4be3a26160a9d7f8b399, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:43.042175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:43.063135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:43.064518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:43.137467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 74/140] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:42 2022] PUT /identity/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555/users/a94c4cb016bc4be3a26160a9d7f8b399/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:43.179950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 67/141] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:43.353038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:43.492577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c29f40-8697-4adf-b97f-2ebe11cccbe8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:43.540247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:43.540967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:43.541252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:43.541524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:43.542650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:43.544105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:43.546414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 75/142] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:43.557436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:43.710093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b5c3d-b516-4bb3-888d-026cb8be69fd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:43.711626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:43.752815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:43.753503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:43.753804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:43.754116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:43.754864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:43.756632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:43.849428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:43.885253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 68/143] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:24:43 2022] POST /identity/v3/users => generated 283 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:43.926919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:44.016240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4ec4-57c1-4b5f-b783-e178602dc9b6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:44.067019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:44.067676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.067960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.068232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.094613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:44.100083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:44.100989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:44.102158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 76/144] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:24:44.107887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:44.237427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d72793-6ffc-485d-a91a-0f269c6efdbd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:44.282355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:44.282902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.283198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.283457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.284136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:44.284533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:44.285998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:44.299606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 69/145] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:44.304913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:44.450907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:44.456402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4e879-566e-4d1e-b8ef-6b0d3a2f7590 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:44.497354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:44.498000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.498334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.498612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] PATH_INFO: `/v3/users/6a2d55b1385f41fa975478f0a05cb942` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.502748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] RBAC: Authorizing `identity:get_user(user_id=6a2d55b1385f41fa975478f0a05cb942)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:44.504297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:44.506242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 77/146] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:44 2022] GET /identity/v3/users/6a2d55b1385f41fa975478f0a05cb942 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:44.511797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:44.540907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:44.627389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773192e6-6c4b-4f2c-a564-0e659c5c6c74 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:44.666278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ab47e-59bf-4907-9bee-956c7b79a54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:44.667067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ab47e-59bf-4907-9bee-956c7b79a54f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.667663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ab47e-59bf-4907-9bee-956c7b79a54f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.667947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ab47e-59bf-4907-9bee-956c7b79a54f None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.669148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ab47e-59bf-4907-9bee-956c7b79a54f None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:44.671241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ab47e-59bf-4907-9bee-956c7b79a54f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:44.673695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 70/147] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:44 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:44.679418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:44.800733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059248f7-ce2d-48d6-8043-61b321bd4ddd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:44.839856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:44.840547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.840827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.841100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/6a2d55b1385f41fa975478f0a05cb942/roles/e4bd5c0c6a9d4d1d97b5fa821356d545` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.843207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e50fa06fb1774ed1b1cc4471b8e86025, user_id=6a2d55b1385f41fa975478f0a05cb942, role_id=e4bd5c0c6a9d4d1d97b5fa821356d545)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:44.844677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:44.927592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66e29-c804-498e-a8db-1875f9b7e392 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.927885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66e29-c804-498e-a8db-1875f9b7e392 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.928183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c66e29-c804-498e-a8db-1875f9b7e392 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.929849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 71/148] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:44.935224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b4f9-041b-4ce8-b6fb-a068e91f776a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:44.935719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b4f9-041b-4ce8-b6fb-a068e91f776a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:44.935998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67b4f9-041b-4ce8-b6fb-a068e91f776a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:44.955399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 78/149] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:44 2022] PUT /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/users/6a2d55b1385f41fa975478f0a05cb942/roles/e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:24:44.965321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cf67b4f9-041b-4ce8-b6fb-a068e91f776a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:44.966625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf67b4f9-041b-4ce8-b6fb-a068e91f776a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:45.056490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 72/150] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:45.361709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:45.395937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:45.427569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42f219-d711-4182-b7c9-4dd5797029a5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:45.428011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42f219-d711-4182-b7c9-4dd5797029a5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:45.428402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42f219-d711-4182-b7c9-4dd5797029a5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:45.429953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 73/151] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:45.435693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:45.449177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:45.449177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:45.465344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:45.466550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:45.489506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:45.508551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf684205-a840-4a36-9bfa-f7ee8efabef0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:45.577762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 74/152] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:45.585069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:45.585728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:45.586004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:45.586273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:45.601374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:45.606192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:45.607039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:45.607940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 79/153] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:45.614745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:45.738964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f69866-9bad-4500-89c4-91bcbc9ad8e0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:45.791652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:45.792272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:45.792572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:45.792838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:45.793518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:45.793815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:45.795233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:45.807749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 75/154] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:45.813693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:45.925351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:45.941232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4221807-6791-4140-8a57-638e6965f2a3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:45.995824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:45.996485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:45.996760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:45.997026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:45.998173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:46.000384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:46.001982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 80/155] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:45 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:46.007615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:46.033965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:46.157241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd9ccd-553d-4724-bf50-5263c7b93fc2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:46.184247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:46.203825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:46.204471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.204748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.205016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.315062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:46.316713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:46.346340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:46.418572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8025584-65e9-4f36-af47-2d662786bb80 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:46.483594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:46.484247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.484549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.484819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.516362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:46.517134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 76/156] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:46.521982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:46.522845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:46.523652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:46.524768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:46.527804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 81/157] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:24:46.528701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:46.564758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:46.715528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87032a-f6d6-41f9-9dd9-5d5641613cdb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:46.731650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:46.748536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803aefdf-865a-4ac8-af7b-830b69364e17 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:46.778125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b165f-75d2-4702-b5b0-570fdcb3c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:46.778806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b165f-75d2-4702-b5b0-570fdcb3c90d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.779026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b165f-75d2-4702-b5b0-570fdcb3c90d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.779293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b165f-75d2-4702-b5b0-570fdcb3c90d None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.781755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b165f-75d2-4702-b5b0-570fdcb3c90d None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:46.783242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b165f-75d2-4702-b5b0-570fdcb3c90d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:46.784923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 77/158] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:46 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:46.786052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0483c-e3cd-452c-8f34-6f633959dca8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.786328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0483c-e3cd-452c-8f34-6f633959dca8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.786595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0483c-e3cd-452c-8f34-6f633959dca8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.787780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 78/159] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:46.801591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd50bd-2ea2-4e2f-b7dd-7600cc835c43 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.801861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd50bd-2ea2-4e2f-b7dd-7600cc835c43 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.802128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd50bd-2ea2-4e2f-b7dd-7600cc835c43 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.803434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 79/160] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:46 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:46.804131 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd50bd-2ea2-4e2f-b7dd-7600cc835c43 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:46.827972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:46.828617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.828892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.829163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.829846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:46.830142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:46.831571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478db2d-212b-4712-bd94-2ac7b2e1e607 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:46.843655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 82/161] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:46.845495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f9211-043a-4d86-a915-e04d900854b2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.845778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f9211-043a-4d86-a915-e04d900854b2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.846049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f9211-043a-4d86-a915-e04d900854b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:46.867565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-491f9211-043a-4d86-a915-e04d900854b2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:46.868782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-491f9211-043a-4d86-a915-e04d900854b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:46.993098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd50bd-2ea2-4e2f-b7dd-7600cc835c43 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:46.994929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 83/162] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:46.996254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:46.996563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:46.996831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.014349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:47.015552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:47.040685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:47.057037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:47.057661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:47.057935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:47.058201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.058934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=a94c4cb016bc4be3a26160a9d7f8b399) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:47.059266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:47.060773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:47.067626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 80/163] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:46 2022] GET /identity/v3/role_assignments?user.id=a94c4cb016bc4be3a26160a9d7f8b399&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:47.072521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:47.087317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 84/164] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:47.142898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:47.232533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b28bee-8321-44a9-a2c8-b27f30c1deb2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:47.275463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:47.276365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:47.276906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:47.277312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] PATH_INFO: `/v3/users/1440567f43e04ab1bb7a5b1e330db669` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.281616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] RBAC: Authorizing `identity:get_user(user_id=1440567f43e04ab1bb7a5b1e330db669)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:47.283254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:47.285099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 81/165] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/users/1440567f43e04ab1bb7a5b1e330db669 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:47.290816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:47.422281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e163ebd-d632-465d-a614-27207fa46597 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:47.458595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:47.459217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:47.459891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:47.460165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.469056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:47.538785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:47.540252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:47.575568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:47.646585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 85/166] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:47.651969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:47.700069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:47.761082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dec5bf-2d2e-4835-be69-47015bea216b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:47.765195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e4f3-aae6-46c2-966c-f4e3d610c53f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:47.776603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:47.804757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:47.805615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:47.806040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:47.806453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.807900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:47.809701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:47.812272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 86/167] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:47.813408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:47.824094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:47.824939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:47.825332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:47.825704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.845470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:47.851590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:47.852919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:47.853939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:47.855614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 82/168] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:47.860147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:47.933840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074fdc-60a8-4078-91f9-fdab74cb061b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:47.942027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:47.972745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:47.973384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:47.974013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:47.974470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:47.986319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:47.991389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:47.992478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:47.993528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:47.995117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 87/169] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:47.995974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:48.046702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51bb00-c697-4ed8-a8c3-337b12738324 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.116879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.117414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.117690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.117963 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.118702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=1440567f43e04ab1bb7a5b1e330db669) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:48.119008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.125069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:48.139652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 83/170] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/role_assignments?user.id=1440567f43e04ab1bb7a5b1e330db669&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:48.141651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:48.179394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c02fbb-1c3c-40f6-a679-750d23e8e35b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.218121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:48.235724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.236374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.236655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.236927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.237608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:48.237909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.239325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:48.248262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 88/171] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:24:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:48.256492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:48.345692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:48.365542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542a72f-015d-4338-aa6a-dc32018ef521 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.464592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.465008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b88f2-369c-4bbd-9abf-bcbfbc7dbaac None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.465725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.466175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.466612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.468393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:48.469146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.470842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:48.484236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 84/172] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:48.489840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:48.511402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.512040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.512317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.512611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.517159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.518626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:48.522752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 89/173] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:48 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:48.529179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:48.640851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:48.646739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a151c8-6939-4980-a137-415e5549cbd9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.691414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:48.697344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543b7ec-4d2f-4242-8754-c97f240b82ec None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.705481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e00cf2-6189-4850-820a-065dd0d12261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.706159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e00cf2-6189-4850-820a-065dd0d12261 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.706446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e00cf2-6189-4850-820a-065dd0d12261 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.706722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e00cf2-6189-4850-820a-065dd0d12261 None admin] PATH_INFO: `/v3/users/a94c4cb016bc4be3a26160a9d7f8b399` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.707974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e00cf2-6189-4850-820a-065dd0d12261 None admin] RBAC: Authorizing `identity:get_user(user_id=a94c4cb016bc4be3a26160a9d7f8b399)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.709685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e00cf2-6189-4850-820a-065dd0d12261 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:48.711710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 85/174] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:48 2022] GET /identity/v3/users/a94c4cb016bc4be3a26160a9d7f8b399 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:48.712903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637f5a3-8613-46d8-801a-5d5139c93548 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.713181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637f5a3-8613-46d8-801a-5d5139c93548 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.713451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637f5a3-8613-46d8-801a-5d5139c93548 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.714877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 86/175] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:48.717959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637f5a3-8613-46d8-801a-5d5139c93548 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:48.740171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf52c70-ff4f-42cc-959b-188363df3830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.740818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52c70-ff4f-42cc-959b-188363df3830 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.741104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52c70-ff4f-42cc-959b-188363df3830 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.741387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf52c70-ff4f-42cc-959b-188363df3830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.815417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf52c70-ff4f-42cc-959b-188363df3830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.816988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf52c70-ff4f-42cc-959b-188363df3830 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:48.840323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637f5a3-8613-46d8-801a-5d5139c93548 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:48.880138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef6f42-cc03-4eb1-84ff-bda55ea6f9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:48.880930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6f42-cc03-4eb1-84ff-bda55ea6f9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.881344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6f42-cc03-4eb1-84ff-bda55ea6f9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.881778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6f42-cc03-4eb1-84ff-bda55ea6f9b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.924356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:48.924939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:48.933470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 90/176] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:24:48.939919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005c1f-ff2d-4bed-af75-9d0250d769c7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:48.939919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005c1f-ff2d-4bed-af75-9d0250d769c7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:48.939919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8005c1f-ff2d-4bed-af75-9d0250d769c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:48.959582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a8005c1f-ff2d-4bed-af75-9d0250d769c7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:48.961003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a8005c1f-ff2d-4bed-af75-9d0250d769c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:48.963626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6f42-cc03-4eb1-84ff-bda55ea6f9b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:48.965714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6f42-cc03-4eb1-84ff-bda55ea6f9b2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:49.033397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 91/177] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:49.034903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c2dfb-b502-4971-ac52-fc2d26198ef5 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.035342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c2dfb-b502-4971-ac52-fc2d26198ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.035747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c2dfb-b502-4971-ac52-fc2d26198ef5 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.037166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 92/178] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:49.042284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72f158-8114-4bb1-99c9-89e18e8a1491 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.042555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72f158-8114-4bb1-99c9-89e18e8a1491 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.042824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72f158-8114-4bb1-99c9-89e18e8a1491 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.048902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 93/179] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:49.049729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72f158-8114-4bb1-99c9-89e18e8a1491 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.084991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 87/180] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:49.086658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.087286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.087767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.107673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:49.109543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:49.160659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72f158-8114-4bb1-99c9-89e18e8a1491 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.193326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:49.203839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768a92e-ecb2-449b-a9ea-5d180a539395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.207004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768a92e-ecb2-449b-a9ea-5d180a539395 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.207004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768a92e-ecb2-449b-a9ea-5d180a539395 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.207004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768a92e-ecb2-449b-a9ea-5d180a539395 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.207004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768a92e-ecb2-449b-a9ea-5d180a539395 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:49.208198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768a92e-ecb2-449b-a9ea-5d180a539395 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:49.212683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 94/181] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:49.212683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b77076-7e53-4c4a-8861-1d9738868ab5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.212683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b77076-7e53-4c4a-8861-1d9738868ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.212683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b77076-7e53-4c4a-8861-1d9738868ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.228606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:49.234275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-88b77076-7e53-4c4a-8861-1d9738868ab5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:49.235598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-88b77076-7e53-4c4a-8861-1d9738868ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:49.236777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 88/182] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:49.243545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.314625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 95/183] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:49.315973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd382c-7abc-49b5-bf03-381d0e2f624b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.316548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd382c-7abc-49b5-bf03-381d0e2f624b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.316987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd382c-7abc-49b5-bf03-381d0e2f624b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.322493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 96/184] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:49.322493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd382c-7abc-49b5-bf03-381d0e2f624b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.375188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8d92-67c7-4e95-81e3-3fd3caf709be None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.412073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3356f-388e-4ade-b819-81c27b865d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.412741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3356f-388e-4ade-b819-81c27b865d8e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.413014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3356f-388e-4ade-b819-81c27b865d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.413332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3356f-388e-4ade-b819-81c27b865d8e None admin] PATH_INFO: `/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.414511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3356f-388e-4ade-b819-81c27b865d8e None admin] RBAC: Authorizing `identity:get_project(project_id=ba2e2b3e0c9b4f37a2290294239e5555)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:49.415969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3356f-388e-4ade-b819-81c27b865d8e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:49.417748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 89/185] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/projects/ba2e2b3e0c9b4f37a2290294239e5555 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:49.418897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.419174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.419450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.438193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:49.439420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:49.454745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd382c-7abc-49b5-bf03-381d0e2f624b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.484025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:49.491315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.491916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.492189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.492499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.493227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] RBAC: Adding query filter params (role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=a94c4cb016bc4be3a26160a9d7f8b399) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:49.493529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:49.495067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:49.501470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 97/186] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/role_assignments?user.id=a94c4cb016bc4be3a26160a9d7f8b399&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:49.502220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.520602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:49.534704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 90/187] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:49.535699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.624983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff8997-86ce-46da-9b4d-32ed76a89186 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.672283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f234e7-e58f-439e-80ba-42f6fec72cf0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.672675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.673267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.673546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.673817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.691411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:49.696836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:49.698571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 98/188] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:49.699421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.729064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.729686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.729961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.730228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.731797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=ba2e2b3e0c9b4f37a2290294239e5555, user.id=a94c4cb016bc4be3a26160a9d7f8b399) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:49.732115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:49.734500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:49.741228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 91/189] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/role_assignments?user.id=a94c4cb016bc4be3a26160a9d7f8b399&scope.project.id=ba2e2b3e0c9b4f37a2290294239e5555&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:49.742037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.771533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:49.832063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:49.853367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589067-db49-411e-bfb3-e3d68524244a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.895408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4de74-1db1-4733-8689-16d5b4b10737 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:49.903687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.904334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.905186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.905461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.907089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:49.909413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:49.910321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 99/190] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:49.915821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:49.944213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:49.945112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:49.945387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:49.945653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:49.963334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:49.965807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:49.966677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:49.967992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 92/191] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:49.968726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.037463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d2d35-77eb-46fe-b808-7607feb8f70c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.053895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.070807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.072274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.072274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.072274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.072593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:50.072915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.074283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.081667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0cb986-5ff8-4274-8be6-04e87157e74f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.083470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 100/192] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:50.085201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.099830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.117298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.117915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.118190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.118452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.129236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:50.134492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:50.135480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:50.136264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 93/193] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:24:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:50.136965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.186627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db821c-3995-4ac8-9093-ec710d53732d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.220325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.220956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.221235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.221505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.222144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.223597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.236318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827236d3-26b8-4e1e-9c20-6cf82aba9972 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.269851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.270468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.270743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.271010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.271665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:50.271962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.273473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.282295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 94/194] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:50.283135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.319345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 101/195] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:24:50 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:50.319345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.338019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.369230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.408964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce60c6-29c6-4c79-9e56-845d5bb26625 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.448591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.449308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.449590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.449864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] PATH_INFO: `/v3/users/b1257bd371af4bcd9fd037b695136ad9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.456030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e5bea-61a3-47ef-9daf-c75a2a62933d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.466133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] RBAC: Authorizing `identity:get_user(user_id=b1257bd371af4bcd9fd037b695136ad9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.469165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.474459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 95/196] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/users/b1257bd371af4bcd9fd037b695136ad9 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:50.474459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.497901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.499567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.499765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.500030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.500884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:50.501186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.502566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.511184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 102/197] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:50.512057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.589410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.598789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf663688-0916-4a30-926d-bd0df7ce4230 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.644840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.645559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.645764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.646033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] PATH_INFO: `/v3/users/6a2d55b1385f41fa975478f0a05cb942` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.647213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] RBAC: Authorizing `identity:get_user(user_id=6a2d55b1385f41fa975478f0a05cb942)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.647479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.648796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.650595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 96/198] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/users/6a2d55b1385f41fa975478f0a05cb942 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:50.652544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.659748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71f5e-8c3c-4418-8451-eeb0d99eee88 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.706410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.707265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.707265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.707668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.773431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff71616-538f-47b7-ba85-2aea312165a9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:50.806526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.808326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.823647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:50.824675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:50.825010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:50.825283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] PATH_INFO: `/v3/users/b1257bd371af4bcd9fd037b695136ad9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:50.827275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] RBAC: Authorizing `identity:get_user(user_id=b1257bd371af4bcd9fd037b695136ad9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:50.829327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:50.831334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 97/199] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/users/b1257bd371af4bcd9fd037b695136ad9 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:50.832191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.876573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.932784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 103/200] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:50.934029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:50.956739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:50.966861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3012f2-5473-4434-8940-1f95c0a65992 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.005769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.006751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.009004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.009004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.063293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8736f1c-b5fd-406a-b0a1-8fba57984a48 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.106775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.108441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.109133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.109412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.109683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.110309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.154523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:51.193214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:51.200686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.203526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.237492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 98/201] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:51.238787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:51.331879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 104/202] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:51.333103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:51.349404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3ef5b-b3f8-49eb-bca9-d23be0d85b43 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.383457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:51.389987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.390631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.390908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.391181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.392880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.394612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.400386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 99/203] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:51 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:51.400386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:51.457436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:51.464109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01c5e2-825c-460e-beba-c29b92fc4f18 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.509856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.510635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.510973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.511278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.513131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.514893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.517217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 105/204] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:51 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:51.517985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:51.536820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71417982-ab3e-4f28-981e-5ff66e43a8cc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.581435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.582190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.582471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.582744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.583889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.585642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.587439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 100/205] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:51 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:51.588318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:51.638741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669d3ca-93f5-4b8f-9b42-6548ea1f0dc8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.665003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:51.679201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.682367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.682367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.682367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.682367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=b1257bd371af4bcd9fd037b695136ad9) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:51.682367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.683533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.690397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 106/206] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:51 2022] GET /identity/v3/role_assignments?user.id=b1257bd371af4bcd9fd037b695136ad9&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:51.691514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:51.705289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8309b7-6dda-412f-a506-57460c5228ed None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.725773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:51.754448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.755077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.755355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.755626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.756558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] RBAC: Adding query filter params (role.id=e4bd5c0c6a9d4d1d97b5fa821356d545, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025, user.id=6a2d55b1385f41fa975478f0a05cb942) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:51.756687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.758171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932eadb-322c-4e39-b551-4c4f33944979 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.770588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 101/207] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:51 2022] GET /identity/v3/role_assignments?user.id=6a2d55b1385f41fa975478f0a05cb942&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:51.829262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33a5e-acb2-4cda-a150-27c5f677b3fd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:51.869103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:51.870115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:51.870591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:51.871056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:51.872010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] RBAC: Adding query filter params (role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=b1257bd371af4bcd9fd037b695136ad9) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:51.872557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:51.874777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2665ca-e02b-4b10-87a1-59db213ec0c6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:51.881344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 107/208] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:51 2022] GET /identity/v3/role_assignments?user.id=b1257bd371af4bcd9fd037b695136ad9&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:52.541196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b0ae7-f5a1-4561-814e-388cd4c9eb68 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:52.541725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b0ae7-f5a1-4561-814e-388cd4c9eb68 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:52.541725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b0ae7-f5a1-4561-814e-388cd4c9eb68 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:52.543079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 102/209] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:52.548968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:52.549253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:52.549524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:52.561612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50f6f7-d169-4f1e-8b39-5ce5fa102b10 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:52.561894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50f6f7-d169-4f1e-8b39-5ce5fa102b10 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:52.562166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50f6f7-d169-4f1e-8b39-5ce5fa102b10 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:52.563753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 103/210] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:52.569718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:52.570001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:52.570277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:52.573146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:52.574346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:52.591465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:52.593063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:52.652454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 108/211] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:52.667280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 104/212] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:52.943656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:52.955618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:52.970932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.044179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.065215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cee7a-cf34-44cc-ada6-70c8a72f7747 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:53.103982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53890a12-d6a9-4f61-91ab-af79754ec0a1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:53.105726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:53.106392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.106735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.107058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.119389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:53.125260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:53.130829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:53.131616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 109/213] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:53.132758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:53.154372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:53.155978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.156391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.156677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.157892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:53.159377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:53.166789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 105/214] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:53.168010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:53.247140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.267117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8ea9a-b0ff-4331-bb4e-f969fb839184 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:53.290707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298741e-4051-47bd-a168-7d80a4812c78 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:53.306162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:53.307074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.307531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.307987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.308958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:53.309533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:53.311995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:53.321323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 110/215] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:53.326898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.327776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:53.331774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:53.333114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.333114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.333114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.340312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:53.340312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:53.436593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad424e7-b8d1-4489-80ef-951d4fac7039 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:53.445773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 106/216] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:24:53 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:24:53.476139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a856fa5-ed59-4d3f-a34b-390f1b0f3e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:53.477131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a856fa5-ed59-4d3f-a34b-390f1b0f3e1a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.477575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a856fa5-ed59-4d3f-a34b-390f1b0f3e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.478118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a856fa5-ed59-4d3f-a34b-390f1b0f3e1a None admin] PATH_INFO: `/v3/users/3df3df4c4d4c454db0504ca5e5d270d3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.495765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a856fa5-ed59-4d3f-a34b-390f1b0f3e1a None admin] RBAC: Authorizing `identity:get_user(user_id=3df3df4c4d4c454db0504ca5e5d270d3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:53.497579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a856fa5-ed59-4d3f-a34b-390f1b0f3e1a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:53.499635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 111/217] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:53 2022] GET /identity/v3/users/3df3df4c4d4c454db0504ca5e5d270d3 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:53.506453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:53.633544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08784ef-c6c5-443c-8c2f-4e249db9c240 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:53.673974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:53.674596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.674876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.675150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.735438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.749675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:53.751202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:53.852662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a2e2-3b04-4360-bb98-5a7435c88495 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.853389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a2e2-3b04-4360-bb98-5a7435c88495 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.854176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21a2e2-3b04-4360-bb98-5a7435c88495 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.855744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 112/218] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:53.863121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c68b7-916a-404a-aded-12d96392f58f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.863421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c68b7-916a-404a-aded-12d96392f58f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.863665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c68b7-916a-404a-aded-12d96392f58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.872711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 107/219] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:24:53.876603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.880163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:53.893173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-009c68b7-916a-404a-aded-12d96392f58f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:53.904228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-009c68b7-916a-404a-aded-12d96392f58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:53.986678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:53.990999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 113/220] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:53.992451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba8005-cb32-4bdd-b776-47c28edc7145 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:53.992736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba8005-cb32-4bdd-b776-47c28edc7145 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:53.993010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba8005-cb32-4bdd-b776-47c28edc7145 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:53.994558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 114/221] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:54.000656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:54.001198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:54.001768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:54.014937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9dfbc-df30-4404-8920-3ad0cdb8b690 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:54.022011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:54.023191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:54.077211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:54.078855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:54.079383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:54.079925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:54.081356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:54.083337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:54.085752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 108/222] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:53 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:54.093190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:54.130246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 115/223] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:54.209498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7b174-fe62-4904-8f56-e43715454cca None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:54.246326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:54.246932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:54.247209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:54.247480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:54.248461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=3df3df4c4d4c454db0504ca5e5d270d3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:54.249008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:54.250472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:54.256201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 109/224] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:54 2022] GET /identity/v3/role_assignments?user.id=3df3df4c4d4c454db0504ca5e5d270d3&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:54.422337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:54.481403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:54.502795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:54.523989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:54.583330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ecf53-a0b6-4799-b1ca-5dfc8ed13601 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:54.655336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:54.656797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:54.658138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:54.661196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:54.679682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:54.687789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:54.688709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:54.689500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:54.692742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e78c8-d363-4640-b665-0e473546d763 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:54.695849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 116/225] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:24:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:54.703148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:54.750762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:54.751857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:54.752492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:54.753059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:54.764331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:54.771035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:54.771907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:54.772699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:54.775854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 110/226] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:54.784471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:54.883394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:54.928782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9d58e-dadd-4f3c-81a6-c11554d44326 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:54.944086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096aadc-aaf2-4853-83ef-5a97da68dfe6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:54.980755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:54.994179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:54.994801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:54.995079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:54.995350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.005124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:55.005124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.005124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.006627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.007240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.007519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.007783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.008537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:55.008847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.010326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.022916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 111/227] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:55.023840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:55.029189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 117/228] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:24:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:55.041224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:55.203269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb133e2-2fa3-4f94-82c1-87955043a391 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:55.229497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459791d-882d-4684-8eab-04b0a26f4174 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:55.265895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.266459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.266736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.267008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] PATH_INFO: `/v3/users/b1257bd371af4bcd9fd037b695136ad9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.268857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] RBAC: Authorizing `identity:get_user(user_id=b1257bd371af4bcd9fd037b695136ad9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.270400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.272283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 112/229] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:55 2022] GET /identity/v3/users/b1257bd371af4bcd9fd037b695136ad9 => generated 283 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:55.280077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:55.297581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.298221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.298498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.298762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] PATH_INFO: `/v3/users/b1257bd371af4bcd9fd037b695136ad9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.301138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] RBAC: Authorizing `identity:get_user(user_id=b1257bd371af4bcd9fd037b695136ad9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.301617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:55.302614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.304426 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 118/230] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:55 2022] GET /identity/v3/users/b1257bd371af4bcd9fd037b695136ad9 => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:55.310288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:55.427784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:55.437199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87336df5-11ec-4642-9b14-4905b9ccf54d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:55.479519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.480168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.480476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.480751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.481914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.484073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.485774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 113/231] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:55 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:55.491806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:55.552082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08ddad-8e19-4a3f-97c4-b2364af046c9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:55.602946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:55.630737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4620cab-88f7-41cd-8aa2-f993c23e3a23 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:55.656547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.657677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.658155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.658624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.659975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.661720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.663593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 119/232] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:55 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:55.673304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:55.681607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d519e-6355-443d-80a9-78d1eb9b9a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.682281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d519e-6355-443d-80a9-78d1eb9b9a93 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.682558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d519e-6355-443d-80a9-78d1eb9b9a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.682830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d519e-6355-443d-80a9-78d1eb9b9a93 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/b1257bd371af4bcd9fd037b695136ad9/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.684826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d519e-6355-443d-80a9-78d1eb9b9a93 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user_id=b1257bd371af4bcd9fd037b695136ad9, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.686957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d519e-6355-443d-80a9-78d1eb9b9a93 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.782809 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd1b27-0653-4b04-b422-9076d60a4806 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:55.783899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 114/233] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:55 2022] PUT /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/b1257bd371af4bcd9fd037b695136ad9/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:55.812827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:55.823439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f1fbd-7891-4dcb-bb58-9d2819d67ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:55.824498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f1fbd-7891-4dcb-bb58-9d2819d67ddc None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.825005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f1fbd-7891-4dcb-bb58-9d2819d67ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.825485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f1fbd-7891-4dcb-bb58-9d2819d67ddc None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/b1257bd371af4bcd9fd037b695136ad9/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.828899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f1fbd-7891-4dcb-bb58-9d2819d67ddc None admin] RBAC: Authorizing `identity:create_grant(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user_id=b1257bd371af4bcd9fd037b695136ad9, role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:55.834693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f1fbd-7891-4dcb-bb58-9d2819d67ddc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:55.849818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9efe13-e03b-4009-a320-526e70bdc84a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.850102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9efe13-e03b-4009-a320-526e70bdc84a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.854826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9efe13-e03b-4009-a320-526e70bdc84a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.854826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 115/234] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:55.860115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551ded0-e246-4fd9-9a85-54d31a2ef8b8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:55.860605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551ded0-e246-4fd9-9a85-54d31a2ef8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:55.861013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551ded0-e246-4fd9-9a85-54d31a2ef8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:55.886661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8551ded0-e246-4fd9-9a85-54d31a2ef8b8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:55.891775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8551ded0-e246-4fd9-9a85-54d31a2ef8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:55.960495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 120/235] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:55 2022] PUT /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/b1257bd371af4bcd9fd037b695136ad9/roles/df3ae0f0715049bf9ef8923719615f92 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:55.968426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 116/236] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:56.152578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece73025-2bb8-462a-a685-75eab4734dbb None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.153556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece73025-2bb8-462a-a685-75eab4734dbb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.154054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece73025-2bb8-462a-a685-75eab4734dbb None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.156862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 121/237] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:56.162724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.163084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.163274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.183702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:56.184913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:56.253537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:56.269709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 117/238] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:56.311255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece73025-2bb8-462a-a685-75eab4734dbb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:56.424718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece73025-2bb8-462a-a685-75eab4734dbb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:56.461375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:56.462352 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.462849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.463338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.464687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:56.466331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:56.468922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 122/239] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:24:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:56.476316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:56.588435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e28269a-3715-4326-9156-89bdfd87e564 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:56.627340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831a07c-d52d-43a7-8a1b-4aa66345798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:56.627953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831a07c-d52d-43a7-8a1b-4aa66345798a None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.628227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831a07c-d52d-43a7-8a1b-4aa66345798a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.628517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831a07c-d52d-43a7-8a1b-4aa66345798a None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.629172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831a07c-d52d-43a7-8a1b-4aa66345798a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:56.630640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831a07c-d52d-43a7-8a1b-4aa66345798a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:56.687163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:56.725882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 118/240] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:24:56 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:24:56.728215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3735c-652a-4681-a63d-1ac3cdd1f190 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.728539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3735c-652a-4681-a63d-1ac3cdd1f190 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.728820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3735c-652a-4681-a63d-1ac3cdd1f190 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.730559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 119/241] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:56.737411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.737727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.738524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.746961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:56.762383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:56.763868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:56.827158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:56.851460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 120/242] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:56.891174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220979bc-c3e2-475a-914f-a58669064909 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:56.947780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:56.948354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:56.948765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:56.949038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:56.969404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:56.974757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:56.975630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:56.977015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 123/243] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:24:56.981794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.091300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ad03d-2ab5-427c-8f6b-1de634863797 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.128999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.129614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.129891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.130158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.130842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:57.131141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.132572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.141542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.142280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 121/244] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:57.149344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.204635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:57.260511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73f989-1729-4496-9dc0-feb4869b619c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.266770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b421fab-ef1e-4867-a373-421cd22b3570 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.277005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:57.299288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.300443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.300868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.301450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] PATH_INFO: `/v3/users/1d46e46c0c5a44b2ade9592374235bc5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.308832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.309929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.310403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.311121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.318750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] RBAC: Authorizing `identity:get_user(user_id=1d46e46c0c5a44b2ade9592374235bc5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.320563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.323062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 122/245] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/users/1d46e46c0c5a44b2ade9592374235bc5 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:57.325012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:57.329084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:57.331059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:57.332134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:24:57.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:24:57.335020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 124/246] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:57.340805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.439525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67132c8-622e-462c-b07d-7c52cd3f5d1a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.442656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:57.449780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991509a7-a439-4e44-b933-b583d5d539c6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.478231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.479088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.479607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.480034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.492008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.492962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.493421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.493863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.494727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:57.495203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.496801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.507097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 125/247] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:57.511946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.550472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:57.557305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.558998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.629688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88d50-ee9e-46b5-b64d-5e249a4b6cc2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.668459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.669498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.670872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.671488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] PATH_INFO: `/v3/users/3df3df4c4d4c454db0504ca5e5d270d3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.673663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] RBAC: Authorizing `identity:get_user(user_id=3df3df4c4d4c454db0504ca5e5d270d3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.675292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.675960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 123/248] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:24:57.677960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 126/249] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/users/3df3df4c4d4c454db0504ca5e5d270d3 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:57.681775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.682592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.684250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:57.798795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830806c6-eaf3-46e6-9289-5bdfe2f55243 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.816642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:57.822312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a9017-1594-430b-80cc-cbcc644f3a6f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:57.873293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.874610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.875059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.875498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.877628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.879285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.884733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 124/250] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:57.893638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:57.902513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:57.905289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:57.905831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:57.906103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:57.907313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:57.911912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:57.917639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 127/251] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:57.920990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:58.036972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013787-354d-4629-b785-ae6d0e58f36b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:58.078882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981392d8-4882-47e1-924b-eeb97b8af1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:58.079511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981392d8-4882-47e1-924b-eeb97b8af1af None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:58.079780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:58.079899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981392d8-4882-47e1-924b-eeb97b8af1af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:58.080053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981392d8-4882-47e1-924b-eeb97b8af1af None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/3df3df4c4d4c454db0504ca5e5d270d3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:58.083132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981392d8-4882-47e1-924b-eeb97b8af1af None admin] RBAC: Authorizing `identity:create_grant(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user_id=3df3df4c4d4c454db0504ca5e5d270d3, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:58.084625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981392d8-4882-47e1-924b-eeb97b8af1af None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:58.140418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e381d-33b1-494b-ad5f-099e31770140 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:58.178971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 128/252] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:24:57 2022] PUT /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/3df3df4c4d4c454db0504ca5e5d270d3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:24:58.215158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:58.215781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:58.216057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:58.216326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:58.217872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] RBAC: Adding query filter params (role.id=e4bd5c0c6a9d4d1d97b5fa821356d545, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=1d46e46c0c5a44b2ade9592374235bc5) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:58.218324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:58.219851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3274-610d-4ab1-950f-df59e24d2583 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:58.226345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 125/253] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:24:57 2022] GET /identity/v3/role_assignments?user.id=1d46e46c0c5a44b2ade9592374235bc5&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:24:58.469575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d99977-a365-43fa-96cf-5e1718d07cd6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:58.470579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d99977-a365-43fa-96cf-5e1718d07cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:58.471060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d99977-a365-43fa-96cf-5e1718d07cd6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:58.472941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 129/254] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:24:58.490258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:58.490554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:58.490841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:58.516525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:58.517716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:58.591433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb724c0-25a2-4f81-b331-7f4bf601c719 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:58.592090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb724c0-25a2-4f81-b331-7f4bf601c719 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:58.592827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb724c0-25a2-4f81-b331-7f4bf601c719 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:58.594364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 130/255] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:24:58.598961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 126/256] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:24:58.599979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:58.600268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:58.600578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:58.621801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:24:58.623695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:24:58.652724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:58.722017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 131/257] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:24:58.899672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:58.986566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:59.035727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d88fc-ae60-4634-a391-dded820a9ef2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:59.085040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:59.085681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:59.085954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:59.086220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:59.099030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:59.103699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:59.104629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:59.105414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 127/258] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:24:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:24:59.112080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:59.282008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:59.286183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86aa168-8c46-4d11-85dc-ee32be04ee2a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:59.342968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:59.343584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:59.343856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:59.344124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:59.346493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:59.346798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:59.348188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:59.365021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 132/259] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:24:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:59.373590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:59.404974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:59.515488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e88e9-8bb4-4d0d-a327-4859061de097 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:59.573451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:59.613521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:59.614270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:59.614550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:59.615004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:59.616893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be09a0-e6f9-4feb-80dc-c2ef29932eb6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:59.645385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:24:59.650248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:24:59.651153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:24:59.651956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:24:59.656717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 128/260] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:24:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:24:59.658776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:59.707567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:59.708224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:59.708706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:59.709023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] PATH_INFO: `/v3/users/b1257bd371af4bcd9fd037b695136ad9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:59.713060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] RBAC: Authorizing `identity:get_user(user_id=b1257bd371af4bcd9fd037b695136ad9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:59.714634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:24:59.717165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 133/261] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:24:59 2022] GET /identity/v3/users/b1257bd371af4bcd9fd037b695136ad9 => generated 283 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:24:59.723389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:24:59.895665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0e06-8b8a-4ed1-9e1e-d3568c9a240c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:59.925207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:24:59.963734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66943760-d55d-4e8d-9e7f-ff54ec9e88cc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:24:59.987948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:24:59.988636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:24:59.988862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:24:59.989131 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:24:59.989807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:24:59.990105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:24:59.991512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:00.013039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 129/262] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:24:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:00.019268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:00.046083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412975ce-0626-4158-bccc-bb1a4eb373fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:00.046706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412975ce-0626-4158-bccc-bb1a4eb373fb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.046984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412975ce-0626-4158-bccc-bb1a4eb373fb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.047252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412975ce-0626-4158-bccc-bb1a4eb373fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.130236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:00.170073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412975ce-0626-4158-bccc-bb1a4eb373fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:00.172574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412975ce-0626-4158-bccc-bb1a4eb373fb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:00.250472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753848-6ce9-4cdc-bfb0-cacad317fd2f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:00.279187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:00.298221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 134/263] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:25:00.299825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d676-1728-4333-ab21-6b1646d03419 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.300265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d676-1728-4333-ab21-6b1646d03419 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.300729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6d676-1728-4333-ab21-6b1646d03419 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.306507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 135/264] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:00.307636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6d676-1728-4333-ab21-6b1646d03419 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:00.321972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85765c9-69da-4efb-aed7-68a29a51fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:00.322630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85765c9-69da-4efb-aed7-68a29a51fc4a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.322928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85765c9-69da-4efb-aed7-68a29a51fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.323313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85765c9-69da-4efb-aed7-68a29a51fc4a None admin] PATH_INFO: `/v3/users/b1257bd371af4bcd9fd037b695136ad9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.324422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85765c9-69da-4efb-aed7-68a29a51fc4a None admin] RBAC: Authorizing `identity:get_user(user_id=b1257bd371af4bcd9fd037b695136ad9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:00.326496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85765c9-69da-4efb-aed7-68a29a51fc4a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:00.334552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 130/265] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:00 2022] GET /identity/v3/users/b1257bd371af4bcd9fd037b695136ad9 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:00.335613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.335877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.336211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.381438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:00.382604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:00.429651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6d676-1728-4333-ab21-6b1646d03419 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:00.468165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd7187-ecad-4b0b-b23a-eea958a6adf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:00.469062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd7187-ecad-4b0b-b23a-eea958a6adf5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.469566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd7187-ecad-4b0b-b23a-eea958a6adf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.470011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd7187-ecad-4b0b-b23a-eea958a6adf5 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.470844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 131/266] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:00.476474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:00.477139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd7187-ecad-4b0b-b23a-eea958a6adf5 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:00.479714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd7187-ecad-4b0b-b23a-eea958a6adf5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:00.485709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 136/267] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:00 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:00.487128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ad3e-542c-430d-9471-0fb78fe12b8b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.488276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ad3e-542c-430d-9471-0fb78fe12b8b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.489180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ad3e-542c-430d-9471-0fb78fe12b8b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.496459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 137/268] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:00.498349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ad3e-542c-430d-9471-0fb78fe12b8b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:00.575198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:00.602468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:00.655155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ad3e-542c-430d-9471-0fb78fe12b8b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:00.674590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7150df1-aea2-48ca-8c22-4f175ba5f35f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:00.698866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:00.699724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.700181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.700627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.702448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] RBAC: Adding query filter params (role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=b1257bd371af4bcd9fd037b695136ad9) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:00.702947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:00.709068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef06d9-2459-47d3-b8bc-1558a8918199 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:00.715225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 138/269] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:00 2022] GET /identity/v3/role_assignments?user.id=b1257bd371af4bcd9fd037b695136ad9&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:00.721849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d018aee-13c6-4bd4-84a6-959e3dedd97a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.722251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d018aee-13c6-4bd4-84a6-959e3dedd97a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.722740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d018aee-13c6-4bd4-84a6-959e3dedd97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.726210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bdc00-5a91-489b-a083-3add3bc01ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:00.727010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bdc00-5a91-489b-a083-3add3bc01ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.727421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bdc00-5a91-489b-a083-3add3bc01ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.727824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bdc00-5a91-489b-a083-3add3bc01ac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.769806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9d018aee-13c6-4bd4-84a6-959e3dedd97a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:00.773160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9d018aee-13c6-4bd4-84a6-959e3dedd97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:00.871550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 139/270] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:00.876373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3278-7be4-45f8-9c61-931aebf1e1e8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:00.876373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3278-7be4-45f8-9c61-931aebf1e1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:00.876373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce3278-7be4-45f8-9c61-931aebf1e1e8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:00.876373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 140/271] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:00.876373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce3278-7be4-45f8-9c61-931aebf1e1e8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:00.885197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bdc00-5a91-489b-a083-3add3bc01ac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:00.886798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bdc00-5a91-489b-a083-3add3bc01ac4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:00.948860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:00.955541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.014658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce3278-7be4-45f8-9c61-931aebf1e1e8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.021899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 132/272] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:01.024063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.024681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.025169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.046536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:01.048137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:01.057613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.058701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.059218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.059685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.061280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:01.063088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:01.068308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 141/273] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:01.069554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.130871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 133/274] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:01.131519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.194103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1a55-768c-4e38-a894-b906015be3b4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.214021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.224545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.233632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a488ff-d1f8-41ac-ac14-eb517c1041a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.234262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a488ff-d1f8-41ac-ac14-eb517c1041a7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.234533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a488ff-d1f8-41ac-ac14-eb517c1041a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.234810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a488ff-d1f8-41ac-ac14-eb517c1041a7 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.235932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a488ff-d1f8-41ac-ac14-eb517c1041a7 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:01.237458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a488ff-d1f8-41ac-ac14-eb517c1041a7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:01.239253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 142/275] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:01.240352 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ad043-aa24-4272-818c-da2cd9943230 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.240637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ad043-aa24-4272-818c-da2cd9943230 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.240904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ad043-aa24-4272-818c-da2cd9943230 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.244464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 143/276] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:01.244464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ad043-aa24-4272-818c-da2cd9943230 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.256853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39dc84-c2f2-4268-b9e0-fab237aa7d7d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.293442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26162388-c651-4dbc-9adb-189dfe0413b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.294138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162388-c651-4dbc-9adb-189dfe0413b7 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.294438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162388-c651-4dbc-9adb-189dfe0413b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.295416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162388-c651-4dbc-9adb-189dfe0413b7 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.296062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162388-c651-4dbc-9adb-189dfe0413b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:01.297560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162388-c651-4dbc-9adb-189dfe0413b7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:01.355926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ad043-aa24-4272-818c-da2cd9943230 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.389109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.389911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.390193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.390464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.400328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:01.406048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:01.406936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:01.407717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 144/277] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:01.408475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.412078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 134/278] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:01 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:01.420778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.420778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.420778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.434059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:01.435979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:01.464551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.524370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 135/279] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:01.524370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.540332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5214be-bb13-4f87-a98d-84da5c0d2c7e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.590861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.591577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.591858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.592127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.593906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=b1257bd371af4bcd9fd037b695136ad9) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:01.594379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:01.596109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:01.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.606799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 145/280] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/role_assignments?user.id=b1257bd371af4bcd9fd037b695136ad9&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:01.606799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.664832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6b3a4-b18c-40f8-989a-495d47dd6e07 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.717644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.718462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.719054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.719307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.736893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:01.743375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:01.744323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:01.745215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:01.746039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 136/281] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:01.754539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.762508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0a998-afb7-4bab-bb24-bcbe1b756639 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.775716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.811010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.811632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.811909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.812183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.812904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:01.813207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:01.814931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:01.828123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 146/282] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:01.829809 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:01.915286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ae611-653d-4e1d-9789-c4b1c884a9b1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.949544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:01.962509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7c4fe-adce-4945-8700-0b04f5e91464 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:01.963080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:01.963735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:01.968527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:01.968527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:01.968527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:01.968527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:01.968527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:01.977353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 137/283] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:01.978297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.001329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.002112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.002391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.002757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.015149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:02.027521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:02.028767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:02.030219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 147/284] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:02.031124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.071466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:02.130806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72ff4-69f8-4186-a769-ec7d73c936a2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.153603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf854da5-5920-4f79-ba65-e2f1393c5d42 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.174404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.175288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.175741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.176163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] PATH_INFO: `/v3/users/afcdda71ff5b439691e12a54ef451cb6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.191962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] RBAC: Authorizing `identity:get_user(user_id=afcdda71ff5b439691e12a54ef451cb6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.194039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.196044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 138/285] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:01 2022] GET /identity/v3/users/afcdda71ff5b439691e12a54ef451cb6 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:02.198130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.199085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.199611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.200105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] PATH_INFO: `/v3/users/1d46e46c0c5a44b2ade9592374235bc5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.202396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] RBAC: Authorizing `identity:get_user(user_id=1d46e46c0c5a44b2ade9592374235bc5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.202541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.204274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.211871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 148/286] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:02 2022] GET /identity/v3/users/1d46e46c0c5a44b2ade9592374235bc5 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:02.213089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.255782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:02.327033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bb7da-c973-4018-bae1-a07f35b411dc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.346224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:02.351463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a828ef3-8a3f-439a-be88-c52e90b32137 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.366713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.367543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.367990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.368305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.369106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:02.369505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.371103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.382615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 139/287] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:02.383832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.399708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.401110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.401632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.402043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.403175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, user.id=afcdda71ff5b439691e12a54ef451cb6, scope.system=all) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:02.403622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.407108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.414043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 149/288] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Aug 24 16:25:02 2022] GET /identity/v3/role_assignments?user.id=afcdda71ff5b439691e12a54ef451cb6&scope.system=all&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:02.414985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.543324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d0ba7-36a7-4980-a3cd-8a2fbecfd1c3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.581761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655c67-4022-4d88-a4af-c360f0e54df8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.583660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:02.587627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.588497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.588935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.589343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] PATH_INFO: `/v3/users/3df3df4c4d4c454db0504ca5e5d270d3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.592243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=3df3df4c4d4c454db0504ca5e5d270d3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.593884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.596127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 150/289] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:02 2022] GET /identity/v3/users/3df3df4c4d4c454db0504ca5e5d270d3 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:02.606179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.647487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.650072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.650357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.650630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.652944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.658631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.660589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 140/290] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:02 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:02.669219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:02.672208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:02.798440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225068-0b61-4cfb-a048-abdaca2b8cd0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.799164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9a5f6-e5e4-4d6f-bc59-55d807f6587e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:02.838604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.839728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.840035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.840305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/1d46e46c0c5a44b2ade9592374235bc5/roles/e4bd5c0c6a9d4d1d97b5fa821356d545` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.845165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] RBAC: Authorizing `identity:create_grant(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user_id=1d46e46c0c5a44b2ade9592374235bc5, role_id=e4bd5c0c6a9d4d1d97b5fa821356d545)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.846809 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.863621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:02.864251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:02.864557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:02.864829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:02.940280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:02.941773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:02.959240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 141/291] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:02 2022] PUT /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/users/1d46e46c0c5a44b2ade9592374235bc5/roles/e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:02.991361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:03.089809 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 151/292] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:25:03.095840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:03.134857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:03.237813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d03507-d5d5-4d68-a683-f761c53c41ff None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:03.276857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d3276-3eb0-48e1-b309-29d6ecc6aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:03.277569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d3276-3eb0-48e1-b309-29d6ecc6aa4d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:03.277854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d3276-3eb0-48e1-b309-29d6ecc6aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:03.278129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d3276-3eb0-48e1-b309-29d6ecc6aa4d None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:03.280056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d3276-3eb0-48e1-b309-29d6ecc6aa4d None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:03.281864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d3276-3eb0-48e1-b309-29d6ecc6aa4d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:03.283854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 142/293] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:03 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:03.289475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:03.402391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437fce1-c4af-4cb1-947e-10068f27ac16 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:03.438314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:03.447842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:03.448479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:03.448761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:03.449034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:03.449772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=3df3df4c4d4c454db0504ca5e5d270d3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:03.450079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:03.452195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00c16e-1028-44f5-bf26-fba28bb0d5ba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:03.458618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 152/294] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:03 2022] GET /identity/v3/role_assignments?user.id=3df3df4c4d4c454db0504ca5e5d270d3&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:03.822710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56bd17-6bf3-4d65-b203-7c9a93b3f954 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:03.823088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56bd17-6bf3-4d65-b203-7c9a93b3f954 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:03.823287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56bd17-6bf3-4d65-b203-7c9a93b3f954 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:03.824901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 143/295] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:03.832386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fbe2c-3687-40cb-a196-450191386531 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:03.832917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fbe2c-3687-40cb-a196-450191386531 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:03.832917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fbe2c-3687-40cb-a196-450191386531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:03.854446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-136fbe2c-3687-40cb-a196-450191386531 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:03.855642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-136fbe2c-3687-40cb-a196-450191386531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:03.939983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 153/296] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:04.054732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba175bf-df1e-46d0-a20f-3d807182badd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.055069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba175bf-df1e-46d0-a20f-3d807182badd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.055289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba175bf-df1e-46d0-a20f-3d807182badd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.056672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 144/297] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:04.062226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.062511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.062780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.083394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:04.084524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:04.239357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 154/298] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:04.243302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba175bf-df1e-46d0-a20f-3d807182badd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:04.345235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:04.350434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba175bf-df1e-46d0-a20f-3d807182badd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:04.388668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:04.389747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.390228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.390724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.401479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:04.407904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:04.408970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:04.409806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:04.412304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 145/299] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:04.418414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:04.476939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:04.519757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:04.524196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864c8a-95e2-45d2-b8bf-db5286a78521 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:04.562175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:04.562892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.563170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.563436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.564100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:04.564421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:04.565778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:04.574926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 155/300] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:04.580945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:04.656959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c54a7-2a32-41bd-a278-568dcc1a03d5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:04.692096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:04.692929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.693207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.693483 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.694688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:04.696147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:04.701454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95312948-342f-41aa-ae11-7d160b423dd4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:04.703475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 146/301] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:04.705126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:04.720269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:04.736102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:04.752557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:04.753739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.754615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.755096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] PATH_INFO: `/v3/users/e27fc98d2e6a48e79268d09c594bf235` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.774941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] RBAC: Authorizing `identity:get_user(user_id=e27fc98d2e6a48e79268d09c594bf235)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:04.777214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:04.779834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 156/302] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:04 2022] GET /identity/v3/users/e27fc98d2e6a48e79268d09c594bf235 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:04.786849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:04.823595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa2fa1-387e-4cbd-bb37-abb5b00f4d91 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:04.859603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b0493-7f6d-4a66-9e40-43ce034bc162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:04.860216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b0493-7f6d-4a66-9e40-43ce034bc162 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.860523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b0493-7f6d-4a66-9e40-43ce034bc162 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.860795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b0493-7f6d-4a66-9e40-43ce034bc162 None admin] PATH_INFO: `/v3/groups` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.861421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b0493-7f6d-4a66-9e40-43ce034bc162 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:04.862866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b0493-7f6d-4a66-9e40-43ce034bc162 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:04.893859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293d566-5ffa-499a-8978-0d3ee7a3a87a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:04.933447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:04.933971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.934248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.934514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.935386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] RBAC: Adding query filter params (role.id=e4bd5c0c6a9d4d1d97b5fa821356d545, user.id=e27fc98d2e6a48e79268d09c594bf235, scope.system=all) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:04.935829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:04.937276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e7ba1-4989-4f33-8a3a-e3d48edba3e7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:04.942024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 157/303] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Aug 24 16:25:04 2022] GET /identity/v3/role_assignments?user.id=e27fc98d2e6a48e79268d09c594bf235&scope.system=all&role.id=e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:04.949952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ccf55-09ea-4327-babe-fb6474478baf None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.950254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ccf55-09ea-4327-babe-fb6474478baf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.950526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ccf55-09ea-4327-babe-fb6474478baf None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.952286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 158/304] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:04.957999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66755a5a-8a36-4586-aea4-385e46c23ec3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:04.958308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66755a5a-8a36-4586-aea4-385e46c23ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:04.958559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66755a5a-8a36-4586-aea4-385e46c23ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:04.976702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 147/305] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Aug 24 16:25:04 2022] POST /identity/v3/groups => generated 229 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:04.985514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-66755a5a-8a36-4586-aea4-385e46c23ec3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:04.986840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66755a5a-8a36-4586-aea4-385e46c23ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:05.027680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:05.079390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 159/306] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:05.217689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09bbe-f549-4675-a457-01bafd698db7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.217883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09bbe-f549-4675-a457-01bafd698db7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.218158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc09bbe-f549-4675-a457-01bafd698db7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.219782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 148/307] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:05.225285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.225586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.225854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.244852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:05.246027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:05.323490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 160/308] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:05.362987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09bbe-f549-4675-a457-01bafd698db7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:05.464632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:05.483122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc09bbe-f549-4675-a457-01bafd698db7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:05.518561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:05.519550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.520030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.520542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.532935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:05.538023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:05.540292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:05.541144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:05.542202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 149/309] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:05.548907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:05.593733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:05.657116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:05.661698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab1161-54ee-44a5-a082-7c09b55bcbb4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:05.699542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:05.700669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.701474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.701963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.702189 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d527b-ed5a-4215-90bd-8d2d30ace20e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:05.703035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:05.703558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:05.705269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:05.719350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 161/310] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:05.725232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:05.746967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:05.748657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.749155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.749663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.765384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:05.773864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:05.774772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:05.775547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:05.776149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 150/311] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:05.783039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:05.795056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:05.859463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e86c8-a74c-45fd-8dbe-1a465026aede None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:05.895383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:05.896840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.897320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.897784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] PATH_INFO: `/v3/users/afcdda71ff5b439691e12a54ef451cb6` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.899263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] RBAC: Authorizing `identity:get_user(user_id=afcdda71ff5b439691e12a54ef451cb6)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:05.900833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:05.907935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 162/312] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:05 2022] GET /identity/v3/users/afcdda71ff5b439691e12a54ef451cb6 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:05.909130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:05.919011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6595d-a7ca-436a-948f-5a7db7e3b004 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:05.948061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:05.963923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:05.964570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:05.964900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:05.965172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:05.965846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:05.966243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:05.968307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:05.979184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 151/313] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:05.985061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:06.044182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0a83d-e0db-44f3-af1b-dbd90535f04c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:06.062363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:06.087463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:06.088905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.089587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.090266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] PATH_INFO: `/v3/system/users/afcdda71ff5b439691e12a54ef451cb6/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:06.093286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=afcdda71ff5b439691e12a54ef451cb6, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:06.095808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:06.100899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bd27b-5664-4338-bec7-819117219a5f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:06.146044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:06.146701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.146972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.147245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] PATH_INFO: `/v3/users/1d46e46c0c5a44b2ade9592374235bc5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:06.148558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] RBAC: Authorizing `identity:get_user(user_id=1d46e46c0c5a44b2ade9592374235bc5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:06.150085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:06.152036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 152/314] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:05 2022] GET /identity/v3/users/1d46e46c0c5a44b2ade9592374235bc5 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:06.158867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:06.187415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 163/315] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Aug 24 16:25:05 2022] PUT /identity/v3/system/users/afcdda71ff5b439691e12a54ef451cb6/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:25:06.281835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fa37a-519f-41c5-8a37-43d85486163e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:06.319030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:06.319567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.319843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.320111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:06.324142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:06.401283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:06.403577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:06.511878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 153/316] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 16:25:06.517503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:06.625134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:06.629111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbb1e2-e011-432b-b160-791464e2760f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:06.666364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61f8d3-4fbd-47ca-a93b-cc87a0bb8daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:06.667048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61f8d3-4fbd-47ca-a93b-cc87a0bb8daf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.667330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61f8d3-4fbd-47ca-a93b-cc87a0bb8daf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.669867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61f8d3-4fbd-47ca-a93b-cc87a0bb8daf None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:06.669867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61f8d3-4fbd-47ca-a93b-cc87a0bb8daf None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:06.670984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61f8d3-4fbd-47ca-a93b-cc87a0bb8daf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:06.672949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 164/317] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:06 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:06.678732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:06.728770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:06.786633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaacc5-e194-4584-a097-79be97bb07e6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:06.826865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:06.827528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.827806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.828073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:06.828850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] RBAC: Adding query filter params (role.id=e4bd5c0c6a9d4d1d97b5fa821356d545, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54, user.id=1d46e46c0c5a44b2ade9592374235bc5) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:06.829155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:06.830730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf5c0e-a4af-476d-903c-6312be6b7ad3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:06.837451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 154/318] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:06 2022] GET /identity/v3/role_assignments?user.id=1d46e46c0c5a44b2ade9592374235bc5&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:06.980700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065845fe-4d88-4fea-b9d1-82be1c19e536 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.981205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065845fe-4d88-4fea-b9d1-82be1c19e536 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.981205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065845fe-4d88-4fea-b9d1-82be1c19e536 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:06.983277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 165/319] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:06.988882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:06.989167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:06.989442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.008983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:07.010179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:07.052441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ee8a9-0d99-46f8-9b85-3a8bda4bde83 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.052441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ee8a9-0d99-46f8-9b85-3a8bda4bde83 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.052441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ee8a9-0d99-46f8-9b85-3a8bda4bde83 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.053490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 166/320] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:07.062069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.062915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.063483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.084365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:07.085671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:07.086231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 155/321] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:07.162171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 167/322] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:07.357248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:07.387397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:07.429963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:07.462921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce717d6d-808d-485c-b62a-a61f54f7f86e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:07.498893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:07.499523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.499799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.500072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.501231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:07.502702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:07.504478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 156/323] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:07.510789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:07.522606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:07.533432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ee0e1-f345-4ce8-a282-e7794f34012d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:07.572580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:07.573249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.573534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.573807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.585660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:07.590384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:07.591560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:07.592396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 168/324] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:07.597266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:07.614480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:07.618465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9de921-57b9-4477-a12e-96baabcb4eb8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:07.653973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:07.655056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.655608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.655885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] PATH_INFO: `/v3/groups` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.656546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:07.658022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:07.702947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ac8c-df76-4dc1-bb33-f0053c72353e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:07.739711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:07.740370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.740879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.741501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.742226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:07.742535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:07.744034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:07.755884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 169/325] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:07.757313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 157/326] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Aug 24 16:25:07 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:07.765988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:07.783323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:07.882270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73388e-e93f-4ef0-9d5c-e093a31ab868 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:07.928832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bd6f3-775e-4f03-ba42-dffdb841fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:07.929469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd6f3-775e-4f03-ba42-dffdb841fabb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:07.929748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd6f3-775e-4f03-ba42-dffdb841fabb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:07.930027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bd6f3-775e-4f03-ba42-dffdb841fabb None admin] PATH_INFO: `/v3/users/e27fc98d2e6a48e79268d09c594bf235` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:07.931206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bd6f3-775e-4f03-ba42-dffdb841fabb None admin] RBAC: Authorizing `identity:get_user(user_id=e27fc98d2e6a48e79268d09c594bf235)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:07.932685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bd6f3-775e-4f03-ba42-dffdb841fabb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:07.934276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 170/327] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:07 2022] GET /identity/v3/users/e27fc98d2e6a48e79268d09c594bf235 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:07.939970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:08.055138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624731e-b501-44eb-bf4e-70a5432db0ae None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:08.092828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37211795-89e6-4329-8461-f25b498a6792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:08.097056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37211795-89e6-4329-8461-f25b498a6792 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:08.097056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37211795-89e6-4329-8461-f25b498a6792 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:08.097056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37211795-89e6-4329-8461-f25b498a6792 None admin] PATH_INFO: `/v3/system/users/e27fc98d2e6a48e79268d09c594bf235/roles/e4bd5c0c6a9d4d1d97b5fa821356d545` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:08.097056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37211795-89e6-4329-8461-f25b498a6792 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e27fc98d2e6a48e79268d09c594bf235, role_id=e4bd5c0c6a9d4d1d97b5fa821356d545)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:08.097056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37211795-89e6-4329-8461-f25b498a6792 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:08.169755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 158/328] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Aug 24 16:25:07 2022] PUT /identity/v3/system/users/e27fc98d2e6a48e79268d09c594bf235/roles/e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:08.274767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7e15f-8b6e-4325-ab41-fb2d3d97718e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:08.275265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7e15f-8b6e-4325-ab41-fb2d3d97718e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:08.275339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7e15f-8b6e-4325-ab41-fb2d3d97718e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:08.276994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 171/329] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:08.283607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:08.284207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:08.284679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:08.305340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:08.306758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:08.326094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:08.381476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 159/330] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:08.671572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7e15f-8b6e-4325-ab41-fb2d3d97718e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:08.796737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:08.833885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7e15f-8b6e-4325-ab41-fb2d3d97718e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:08.875106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:08.875737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:08.876012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:08.876278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:08.887021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:08.892920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:08.893866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:08.894646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 172/331] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:08.901597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:09.018034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2835c-0cf9-4e66-bf35-adc6f0e509b1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:09.055979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:09.056631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:09.056909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:09.057179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:09.057855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:09.058151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:09.059549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:09.071859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 160/332] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:09.077887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:09.221215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb912e-3212-4fc0-b2dd-b6b488ea2e15 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:09.266155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52b094-ee74-4c9b-a720-ae0a59aece9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:09.266793 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52b094-ee74-4c9b-a720-ae0a59aece9a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:09.267073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52b094-ee74-4c9b-a720-ae0a59aece9a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:09.267337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52b094-ee74-4c9b-a720-ae0a59aece9a None admin] PATH_INFO: `/v3/users/afcdda71ff5b439691e12a54ef451cb6` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:09.268659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52b094-ee74-4c9b-a720-ae0a59aece9a None admin] RBAC: Authorizing `identity:get_user(user_id=afcdda71ff5b439691e12a54ef451cb6)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:09.270627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52b094-ee74-4c9b-a720-ae0a59aece9a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:09.272004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 173/333] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:09 2022] GET /identity/v3/users/afcdda71ff5b439691e12a54ef451cb6 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:09.278003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:09.313951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:09.414326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff450bf-f87b-4137-af65-57ea1754d113 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:09.452919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:09.453526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:09.453799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:09.454064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:09.454792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] RBAC: Adding query filter params (role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, user.id=afcdda71ff5b439691e12a54ef451cb6, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:09.455096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:09.456561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6300ba8-79f0-45be-8907-e3dc2fc5cb7c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:09.469758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 161/334] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Aug 24 16:25:09 2022] GET /identity/v3/role_assignments?user.id=afcdda71ff5b439691e12a54ef451cb6&scope.system=all&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:10.601888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d3d33-460a-43c0-8712-71f901a73111 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:10.602392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d3d33-460a-43c0-8712-71f901a73111 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:10.602491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d3d33-460a-43c0-8712-71f901a73111 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:10.603755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 174/335] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:10.610592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d4569-7788-4e43-969c-f8360299e45f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:10.611047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d4569-7788-4e43-969c-f8360299e45f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:10.611119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d4569-7788-4e43-969c-f8360299e45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:10.645859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f55d4569-7788-4e43-969c-f8360299e45f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:10.647068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f55d4569-7788-4e43-969c-f8360299e45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:10.754293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 162/336] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:11.066548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7b978-1460-405d-8e4e-8f1da704668e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.066934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7b978-1460-405d-8e4e-8f1da704668e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.067110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7b978-1460-405d-8e4e-8f1da704668e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.068665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 175/337] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:11.074062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ae1a-34f8-4351-9477-cf1f328b2e20 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.074347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ae1a-34f8-4351-9477-cf1f328b2e20 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.074619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7ae1a-34f8-4351-9477-cf1f328b2e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.083113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:11.095577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-64a7ae1a-34f8-4351-9477-cf1f328b2e20 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:11.096781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-64a7ae1a-34f8-4351-9477-cf1f328b2e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:11.171091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 163/338] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:11.349797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7b978-1460-405d-8e4e-8f1da704668e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:11.357745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2b652-bc8a-4fbc-a49d-81cc3f35d13b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.358026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2b652-bc8a-4fbc-a49d-81cc3f35d13b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.358298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2b652-bc8a-4fbc-a49d-81cc3f35d13b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.359941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 164/339] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:11.362540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:11.365897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad881bb-f170-47d0-a418-8dd13efea77c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.366185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad881bb-f170-47d0-a418-8dd13efea77c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.366461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad881bb-f170-47d0-a418-8dd13efea77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.393104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3ad881bb-f170-47d0-a418-8dd13efea77c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:11.394319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3ad881bb-f170-47d0-a418-8dd13efea77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:11.490699 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7b978-1460-405d-8e4e-8f1da704668e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:11.528769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:11.529399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.529672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.529937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.541564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 165/340] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:11.542922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:11.543335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed48aa2-c3fa-4334-9442-9844d303efa1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.544149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed48aa2-c3fa-4334-9442-9844d303efa1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.544808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed48aa2-c3fa-4334-9442-9844d303efa1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.546396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 166/341] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:11.551053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:11.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:11.552744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 176/342] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:11.555008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:11.555452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.555834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.556201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.593378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:11.594849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:11.632743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:11.664838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac16f9-1ddf-4dcf-95d6-5b507868a640 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:11.701358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:11.702554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:11.703001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.703156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.703426 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.704094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:11.704419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:11.706506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:11.717865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 177/343] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:11.718900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:11.730023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 167/344] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:11.730992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:11.853248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c96e7-c789-4805-8a58-bcd2ece10269 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:11.886173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:11.900323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:11.902970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.902970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.902970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:11.907659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fea4d-d59a-4787-a712-cd027c297915 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:11.928239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:11.933798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:11.934704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:11.935550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:11.936145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 178/345] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:11.949545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:11.954573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:11.955006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:11.955283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:11.955551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.034975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.053201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.054693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.088271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5cd3a-b591-4c1e-965f-cdd5c9828580 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.149260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.149796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.150068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.150336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.151001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:12.151296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.155052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.175813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 179/346] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:12.176751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.267157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 168/347] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:12.268544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.283468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.345784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615e322-b752-49f8-b7cb-fe433b6ed70c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.373858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.382754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aceb98-46db-47a5-838b-6e8c92580fd7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.385159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.385981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.388964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.388964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.398492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:12.406300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:12.407245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 16:25:12.408055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:12.410520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 180/348] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:12.411487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.425120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.425711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.425987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.426253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.499193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.500975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.526187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324483d2-37e9-4384-8ae4-cc31b7ab2d30 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.540945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.565229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.566149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.566569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.567020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.581474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:12.587491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:12.588486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:12.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:12.592399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 181/349] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:12.593420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.603809 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.614339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 169/350] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:12.615194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.708740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62c475-4170-4b1c-9d3f-0ad330c5eb3c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.725004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c6ebf-937c-461c-a222-f863b1b2c226 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.752435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.753083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.753409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.753686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.754893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.756368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.758026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 182/351] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:12.758803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.763738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.765401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.765677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.765943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.766616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:12.766915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.768314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.777031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 170/352] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:12.777766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.814529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.847009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:12.888633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26311475-c014-469a-a3ce-fa08571bcf7b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.896569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb982287-5af4-40ff-b08a-e8ea9491cfa7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:12.932541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.936431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.936431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.936431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.936431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:12.936431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.936816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8867d1a-13ac-49f0-b928-615e193bc02f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.943159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:12.943792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.944067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.944332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.945568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:12.948395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 183/353] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:12.948514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:12.948587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947d685-f3cb-4bb7-a892-c180ae7cf6c9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.948587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947d685-f3cb-4bb7-a892-c180ae7cf6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.948707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947d685-f3cb-4bb7-a892-c180ae7cf6c9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.950728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 184/355] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:12.951964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b5fb-ddec-4f66-a862-de83c85bb91c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:12.952190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 171/355] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:12.952292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b5fb-ddec-4f66-a862-de83c85bb91c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:12.952514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56b5fb-ddec-4f66-a862-de83c85bb91c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:12.953095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:12.953824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 185/356] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:12.954378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b5fb-ddec-4f66-a862-de83c85bb91c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.078287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803e705-97cc-4d16-a7da-a73e55386902 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.083018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56b5fb-ddec-4f66-a862-de83c85bb91c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.085659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.104108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.120185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa1122-50e6-402f-997e-3b28c41b6826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.120847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1122-50e6-402f-997e-3b28c41b6826 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.121122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1122-50e6-402f-997e-3b28c41b6826 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.123940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa1122-50e6-402f-997e-3b28c41b6826 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.124089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.124828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.126237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.126237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] PATH_INFO: `/v3/users/e27fc98d2e6a48e79268d09c594bf235` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.128433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] RBAC: Authorizing `identity:get_user(user_id=e27fc98d2e6a48e79268d09c594bf235)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.133562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.133665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1122-50e6-402f-997e-3b28c41b6826 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.133740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 186/357] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/users/e27fc98d2e6a48e79268d09c594bf235 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:13.133740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.133856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa1122-50e6-402f-997e-3b28c41b6826 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.136209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 172/358] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:12 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:13.138399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c3634-ac5a-49e3-9618-eeab6ad2e742 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.138665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c3634-ac5a-49e3-9618-eeab6ad2e742 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.138936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c3634-ac5a-49e3-9618-eeab6ad2e742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.160315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fd1c3634-ac5a-49e3-9618-eeab6ad2e742 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:13.161905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd1c3634-ac5a-49e3-9618-eeab6ad2e742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:13.248811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 173/359] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:13.250066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.250394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.250668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.259904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504789-a4a8-4009-932f-a1502f099b9b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.269986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:13.270579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:13.314620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.315237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.315510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.315774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.357851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 174/360] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:13.359001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.379390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.414530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.416091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.417761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.494443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b216b-ee7c-4b1c-ab25-f2a3327c8ea0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.543501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.545319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.545600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.545873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.549876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.551413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.553415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 175/361] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:13.554198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.555813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 187/362] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:13.560789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.653427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.682135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3a3fc-ab19-49bb-8e60-16c227d86e48 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.693642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12380cd1-32b9-413a-884e-a1f58ae5c7bc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.698647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.724599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.725219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.725754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.725754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.726481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] RBAC: Adding query filter params (role.id=e4bd5c0c6a9d4d1d97b5fa821356d545, user.id=e27fc98d2e6a48e79268d09c594bf235, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:13.726791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.728243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.736690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.737304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.737586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.737860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.738599 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:13.738908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.740392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.745019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 176/363] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/role_assignments?user.id=e27fc98d2e6a48e79268d09c594bf235&scope.system=all&role.id=e4bd5c0c6a9d4d1d97b5fa821356d545 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:13.745663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.753514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 188/364] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:13.754761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.869037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4987bf1-620c-4049-86ec-7ce31430a402 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.890163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff882b-95f6-4ea4-9547-f9700c9f24e0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:13.918363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.919723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.920043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.920316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.932243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:13.936027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:13.936673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:13.936948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:13.937216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:13.937948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:13.938252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:13.938353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:13.939179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:13.940021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 177/365] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:13.941152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:13.941219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.945686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:13.950632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 189/366] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:13.951605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:13.987421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.096674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cc4ab-7069-4613-a346-ecd6cd933138 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.097569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee4db3-ac00-42ba-ac6d-637470128b85 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.136261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.136934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.137209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.138158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.142263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.142971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.143254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.143529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.144761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.146363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.148733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 178/367] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:14.149535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:14.151332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:14.158700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:14.159743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:14.160600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:14.164502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 190/368] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:14.165555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:14.232765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.265054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.268986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d044026-c389-4fbb-a5bb-5e35cdcef0c9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.281072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79864018-0119-47d2-9c2f-f4bf52ceada7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.314257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.314881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.315155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.315416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.316094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:14.316423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.317832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.318753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.319481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.319760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.320131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.321303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.322734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.324433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 191/369] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:14.325822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:14.329111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 179/370] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:14.330989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:14.442264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47a13-303c-42e3-8af7-ba0f36e434ea None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.445944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdd2f2-7e05-4d75-b74a-89183785809e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.474334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.491289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.491882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.492166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.492476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.493149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:14.493451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.494835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.495229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.495946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.496521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.500367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.500367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:14.500367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.500367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.508830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 192/371] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:14.509690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:14.514744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 180/372] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:14.519072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:14.558575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.651263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350d2c3-eced-4d7e-880b-839f0eab69da None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.658512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58773e0-afe1-4e7d-8a03-ef9b47dd9bf7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:14.705308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.705654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:14.705938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.706127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:14.706278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.706421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:14.706569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.706713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:14.791580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.792857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:14.793316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.794980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:14.805276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.846811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:14.929130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 193/373] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:14.938902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:15.027068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 181/374] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:15.034419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:15.101509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:15.126525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd02f7-5c4a-4569-b348-75df2c35d52f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:15.201626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:15.202649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:15.203136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:15.203612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:15.206930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:15.208650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:15.210916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 194/375] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:14 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:15.211098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950f9fc-3b33-4fe0-9260-f05b27239545 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:15.218059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:15.239843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:15.249993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:15.250911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:15.251327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:15.251776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] PATH_INFO: `/v3/projects/86cc49d474614954959fd1fd2c69fbe7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:15.252930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] RBAC: Authorizing `identity:get_project(project_id=86cc49d474614954959fd1fd2c69fbe7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:15.254402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:15.256054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 182/376] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:15 2022] GET /identity/v3/projects/86cc49d474614954959fd1fd2c69fbe7 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:15.261692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:15.374349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4d27a-c463-4fc0-b1b0-1f392fedb4db None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:15.383856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e6de0-5a0b-41f8-94ac-ebd9d923fdf0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:15.439586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:15.440482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:15.442003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:15.442910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:15.444779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:15.447117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:15.448432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:15.449228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:15.451890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:15.452355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] PATH_INFO: `/v3/groups/20e46fa44a6d4b0f920653feb35bb43e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:15.454967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 195/377] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:15 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:15.458569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:15.462639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] RBAC: Authorizing `identity:get_group(group_id=20e46fa44a6d4b0f920653feb35bb43e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:15.464291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:15.467559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 183/378] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:15 2022] GET /identity/v3/groups/20e46fa44a6d4b0f920653feb35bb43e => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:15.478312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:15.512141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:15.603633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:15.627023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9947843-b3d5-474a-b2d7-d579dea008e8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:15.650331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d650-e1a1-48f7-9465-b15cb24cbc48 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:15.675107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:15.675714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:15.675990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:15.676263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:15.677022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:15.677329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:15.678778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1858a-e554-4d91-baf0-1fb02584a8ae None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:15.685248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 196/379] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:15 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:15.708685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:15.709638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:15.710142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:15.710613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:15.711564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] RBAC: Adding query filter params (group.id=20e46fa44a6d4b0f920653feb35bb43e, role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=86cc49d474614954959fd1fd2c69fbe7) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:15.712072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:15.713788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f542c-7039-4e54-95c2-56842d063951 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:15.722620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 184/380] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:15 2022] GET /identity/v3/role_assignments?group.id=20e46fa44a6d4b0f920653feb35bb43e&scope.project.id=86cc49d474614954959fd1fd2c69fbe7&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:16.245565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e63a4-8bbe-4007-ab3c-68910079ce5a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:16.246059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e63a4-8bbe-4007-ab3c-68910079ce5a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:16.246402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e63a4-8bbe-4007-ab3c-68910079ce5a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:16.248451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 197/381] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:16.253601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07fd64-433d-4832-9a53-422510d352c4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:16.254035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07fd64-433d-4832-9a53-422510d352c4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:16.254228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07fd64-433d-4832-9a53-422510d352c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:16.283135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-be07fd64-433d-4832-9a53-422510d352c4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:16.284371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-be07fd64-433d-4832-9a53-422510d352c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:16.362433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 185/382] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:16.595372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d4b7-fb96-454f-9b85-f346d526c37f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:16.595720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d4b7-fb96-454f-9b85-f346d526c37f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:16.595834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90d4b7-fb96-454f-9b85-f346d526c37f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:16.597461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 198/383] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:16.602487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df3c07-7cb0-4199-86c3-b8129103dfa3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:16.602983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df3c07-7cb0-4199-86c3-b8129103dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:16.603471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df3c07-7cb0-4199-86c3-b8129103dfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:16.632370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b4df3c07-7cb0-4199-86c3-b8129103dfa3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:16.633901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b4df3c07-7cb0-4199-86c3-b8129103dfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:16.711924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:16.743151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 186/384] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:16.897882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d4b7-fb96-454f-9b85-f346d526c37f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:16.994690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.009085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90d4b7-fb96-454f-9b85-f346d526c37f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.025860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620da21-0c8f-4784-b5c1-b94f62f3e1ca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.026481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620da21-0c8f-4784-b5c1-b94f62f3e1ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.026955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620da21-0c8f-4784-b5c1-b94f62f3e1ca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.029397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 187/385] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:17.034990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.035531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.036000 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.059960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.060953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.061458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.061955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.068536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:17.071938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:17.084199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:17.090866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:17.091842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:17.092679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 199/386] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:17.093341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.164559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 188/387] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:17.165816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.244823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37659a-f391-4771-aa87-4b0336bcd93e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.282247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.282860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.283131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.283405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.294936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:17.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:17.301543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:17.302319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 200/388] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:17.311051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.311275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bc899-5a2e-404d-89be-0fa3d4a869ed None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.329013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.354151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.354792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.355035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.355380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.355715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.360363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:17.360363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:17.360363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:17.367838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 189/389] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:17.374781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.479772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3088e-1bac-42a0-bf03-a10ddbae7642 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.505128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23cc30-4e85-4287-b2a1-6ca159957f81 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.516224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.516895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.517177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.517456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.518592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:17.520058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:17.521978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 190/390] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:17.522751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.540067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.540698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.540974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.541242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.541904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:17.542201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:17.544204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:17.552794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 201/391] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:17.553436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.580104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.653256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.659751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c405f3c-3ec1-4d85-b232-b332fd578ad4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.690176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42820e-1937-433b-9a97-3915500eaff1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.706076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.706993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.707408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.707936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.723934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:17.726792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.727424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.727704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.727973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.729147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:17.730084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:17.731516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:17.732326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:17.732950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:17.733018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 191/392] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:17.733082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 202/393] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:17.733146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.737874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.844589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6e0ea-06fe-4ad7-88f2-1b521061991b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.846403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.855622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807054-da3b-4c47-8a30-7bd25b412006 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:17.888305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.891375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.891375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.891375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.891375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:17.894584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:17.898945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 192/394] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:17.899935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:17.902236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:17.903528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:17.904041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:17.904446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:17.905375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:17.905677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:17.907054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:17.912822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:17.917315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 203/395] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:17.918358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.025055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005437d-04a3-41f9-ba77-9bfe4f4585d0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.035381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df695d30-326b-454e-8f8b-5d04dff38a65 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.071559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768456be-a2a3-4faf-be82-5f5ad2531bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.072595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768456be-a2a3-4faf-be82-5f5ad2531bcf None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.072952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768456be-a2a3-4faf-be82-5f5ad2531bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.073261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768456be-a2a3-4faf-be82-5f5ad2531bcf None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/groups/a155663a0d334bd88ae938e0c2297a9e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.075209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768456be-a2a3-4faf-be82-5f5ad2531bcf None admin] RBAC: Authorizing `identity:create_grant(project_id=e50fa06fb1774ed1b1cc4471b8e86025, group_id=a155663a0d334bd88ae938e0c2297a9e, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:18.075702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.076329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.076634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.076728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768456be-a2a3-4faf-be82-5f5ad2531bcf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:18.076904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.078030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:18.079455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:18.081146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 204/396] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:17 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:18.081904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.131545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 193/397] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Aug 24 16:25:17 2022] PUT /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/groups/a155663a0d334bd88ae938e0c2297a9e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:18.132858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf3254-6546-426e-ba44-44e397196846 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.133142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf3254-6546-426e-ba44-44e397196846 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.133451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf3254-6546-426e-ba44-44e397196846 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.134738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 194/398] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:18.135453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf3254-6546-426e-ba44-44e397196846 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.163435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:18.188759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:18.214199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26f377-6e62-4e53-8deb-d711b833c406 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.261655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb30388-47fe-479f-8ea6-c1bf58362046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.262320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb30388-47fe-479f-8ea6-c1bf58362046 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.262595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb30388-47fe-479f-8ea6-c1bf58362046 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.262870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb30388-47fe-479f-8ea6-c1bf58362046 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.264161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb30388-47fe-479f-8ea6-c1bf58362046 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:18.265668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb30388-47fe-479f-8ea6-c1bf58362046 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:18.267454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 205/399] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:18 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:18.269893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf3254-6546-426e-ba44-44e397196846 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.270232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f90e0f-da26-42d8-9bc8-192cf3b1dbb3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.270944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f90e0f-da26-42d8-9bc8-192cf3b1dbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.271613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f90e0f-da26-42d8-9bc8-192cf3b1dbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.293202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f2f90e0f-da26-42d8-9bc8-192cf3b1dbb3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:18.295257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f2f90e0f-da26-42d8-9bc8-192cf3b1dbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:18.315279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a02f6-efc3-46f3-8122-8f2f2d24e180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.315972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a02f6-efc3-46f3-8122-8f2f2d24e180 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.316257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a02f6-efc3-46f3-8122-8f2f2d24e180 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.316604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a02f6-efc3-46f3-8122-8f2f2d24e180 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/groups/a155663a0d334bd88ae938e0c2297a9e/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.318916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a02f6-efc3-46f3-8122-8f2f2d24e180 None admin] RBAC: Authorizing `identity:create_grant(project_id=e50fa06fb1774ed1b1cc4471b8e86025, group_id=a155663a0d334bd88ae938e0c2297a9e, role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:18.320392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a02f6-efc3-46f3-8122-8f2f2d24e180 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:18.367708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 206/400] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:18.369042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a2f3-c6dc-4c66-9330-284f84ebe0be None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.369313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a2f3-c6dc-4c66-9330-284f84ebe0be None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.369583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a2f3-c6dc-4c66-9330-284f84ebe0be None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.372407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 207/401] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:18.372407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a2f3-c6dc-4c66-9330-284f84ebe0be None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.383361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 195/402] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Aug 24 16:25:18 2022] PUT /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025/groups/a155663a0d334bd88ae938e0c2297a9e/roles/df3ae0f0715049bf9ef8923719615f92 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:18.384989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.385494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.385928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.419783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:18.423998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:18.509035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:18.548296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 196/403] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:18.568715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a2f3-c6dc-4c66-9330-284f84ebe0be None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.604559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.605293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.605570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.606424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.607559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:18.609112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:18.610955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 208/404] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:18 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:18.616579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.640557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:18.708728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.741766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5ef4-e58c-4571-9530-94278d4de745 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.813711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.814305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.814580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.814853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/groups/a155663a0d334bd88ae938e0c2297a9e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.819791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54, group_id=a155663a0d334bd88ae938e0c2297a9e, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:18.821275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:18.868739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141728c-8950-44d1-80e1-62ae93e245c1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:18.912773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:18.913463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:18.913745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:18.914075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:18.916263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:18.929890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 197/405] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Aug 24 16:25:18 2022] PUT /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/groups/a155663a0d334bd88ae938e0c2297a9e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:18.932193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:18.935622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:18.941917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:18.942890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:18.943721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:18.948110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 209/406] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:18.949365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.049244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:19.071852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a9c8d-60c2-4b52-844a-65960ceb51a5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.085912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76aa7cb-2243-400d-895d-fe530cc8bcd0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.114433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.115141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.115433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.115739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.131290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:19.133272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.134467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.134467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.134994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.135870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:19.136394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:19.137891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:19.138757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:19.139544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:19.141597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.145043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 198/407] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:19.147522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.151853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 210/408] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:19.172715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.250592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:19.313408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e45d08-3bb1-41a1-8079-16c902654c25 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.343683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6cb15-0c53-4c58-b896-ba565c33991e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.354759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.355378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.355660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.356124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.357056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:19.357365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.358882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.368184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 199/409] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:19.377448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.378063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.378815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.379092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.379360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.380182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:19.380506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.381936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.383774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 211/410] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:19 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:19.389750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.521938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3303479-c53b-4f21-bcf3-70da99fe8b70 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.540534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f706b-0437-4813-aa61-342119d77aac None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.581870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:19.599703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.599703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.599703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.600083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.600182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.600304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.600304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.600482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.600558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] PATH_INFO: `/v3/groups/20e46fa44a6d4b0f920653feb35bb43e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.600558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] RBAC: Authorizing `identity:get_group(group_id=20e46fa44a6d4b0f920653feb35bb43e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.600707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.600784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.601798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 212/411] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:19 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:19.602744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 200/412] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:19 2022] GET /identity/v3/groups/20e46fa44a6d4b0f920653feb35bb43e => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:19.613366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.615537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.740687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:19.801099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0117e7-8210-4949-bcd2-f1a3baa693de None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.815449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb5c28-fc83-4e0f-98df-4c265aabb789 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:19.866896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.867457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.867737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.868010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] PATH_INFO: `/v3/projects/86cc49d474614954959fd1fd2c69fbe7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.869290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] RBAC: Authorizing `identity:get_project(project_id=86cc49d474614954959fd1fd2c69fbe7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.870753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.873047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 201/413] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:19 2022] GET /identity/v3/projects/86cc49d474614954959fd1fd2c69fbe7 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:19.883609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:19.889470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4fe87-dc30-4d05-9c2c-d766034cca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:19.890225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4fe87-dc30-4d05-9c2c-d766034cca06 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:19.896373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4fe87-dc30-4d05-9c2c-d766034cca06 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:19.896373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4fe87-dc30-4d05-9c2c-d766034cca06 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/groups/a155663a0d334bd88ae938e0c2297a9e/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:19.896605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4fe87-dc30-4d05-9c2c-d766034cca06 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54, group_id=a155663a0d334bd88ae938e0c2297a9e, role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:19.899436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4fe87-dc30-4d05-9c2c-d766034cca06 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:19.997925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 213/414] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Aug 24 16:25:19 2022] PUT /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54/groups/a155663a0d334bd88ae938e0c2297a9e/roles/df3ae0f0715049bf9ef8923719615f92 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:20.016680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c792bb-51c4-4c44-98eb-89372a55842b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:20.063886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ea56f-53c1-4e76-9527-1a3f7d1d711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:20.065128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ea56f-53c1-4e76-9527-1a3f7d1d711e None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:20.065693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ea56f-53c1-4e76-9527-1a3f7d1d711e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:20.066002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ea56f-53c1-4e76-9527-1a3f7d1d711e None admin] PATH_INFO: `/v3/projects/86cc49d474614954959fd1fd2c69fbe7/groups/20e46fa44a6d4b0f920653feb35bb43e/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:20.069780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ea56f-53c1-4e76-9527-1a3f7d1d711e None admin] RBAC: Authorizing `identity:create_grant(project_id=86cc49d474614954959fd1fd2c69fbe7, group_id=20e46fa44a6d4b0f920653feb35bb43e, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:20.071274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ea56f-53c1-4e76-9527-1a3f7d1d711e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:20.158247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:20.165195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 202/415] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Aug 24 16:25:19 2022] PUT /identity/v3/projects/86cc49d474614954959fd1fd2c69fbe7/groups/20e46fa44a6d4b0f920653feb35bb43e/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:25:20.287377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925b65-41f0-4e4f-90d7-49b26b921830 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:20.287733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925b65-41f0-4e4f-90d7-49b26b921830 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:20.287934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf925b65-41f0-4e4f-90d7-49b26b921830 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:20.289385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 214/416] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:20.296017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf61b7d-a9af-4a9e-af9e-aae82a0c91c2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:20.296327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf61b7d-a9af-4a9e-af9e-aae82a0c91c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:20.296607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf61b7d-a9af-4a9e-af9e-aae82a0c91c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:20.320480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fcf61b7d-a9af-4a9e-af9e-aae82a0c91c2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:20.321765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fcf61b7d-a9af-4a9e-af9e-aae82a0c91c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:20.403574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 203/417] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:20.742506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd5c6-94d8-4b8a-869e-6b21c53f6c36 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:20.743054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd5c6-94d8-4b8a-869e-6b21c53f6c36 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:20.743325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dd5c6-94d8-4b8a-869e-6b21c53f6c36 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:20.744982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 215/418] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:20.750882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:20.751176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:20.752028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:20.774100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd5c6-94d8-4b8a-869e-6b21c53f6c36 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:20.778817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:20.780281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:20.838984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:20.862611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 204/419] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:20.900966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dd5c6-94d8-4b8a-869e-6b21c53f6c36 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:20.938225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:20.939176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:20.939633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:20.940085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:20.951149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:20.956226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:20.957185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:20.958039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:20.960224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 216/420] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:20.966144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.033137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:21.096097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae51f5e-e8b1-49a8-a0f0-818a136c89bd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:21.131440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:21.132053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.132327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.132625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.133359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:21.133660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:21.135057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:21.144763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 205/421] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:21.150408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.256451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c699f4-97bd-4fda-ad54-9767d8c76d46 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:21.270552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:21.292468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c72a6-6c6b-4fb8-aa24-8deed76d7eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:21.293026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c72a6-6c6b-4fb8-aa24-8deed76d7eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.293837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c72a6-6c6b-4fb8-aa24-8deed76d7eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.294103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c72a6-6c6b-4fb8-aa24-8deed76d7eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.334778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.367927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c72a6-6c6b-4fb8-aa24-8deed76d7eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:21.369430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c72a6-6c6b-4fb8-aa24-8deed76d7eb6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:21.462695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa27519-1676-4a03-8a36-8547b6b0bfd2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:21.481528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 217/422] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:21.483104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cb347-f2f6-4569-b198-1becbdd7d65f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.483539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cb347-f2f6-4569-b198-1becbdd7d65f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.483948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cb347-f2f6-4569-b198-1becbdd7d65f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.485648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 218/423] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:21.486507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cb347-f2f6-4569-b198-1becbdd7d65f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.489932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:21.505243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:21.507453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4bc11-0097-4298-96da-cb59e093ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:21.508384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4bc11-0097-4298-96da-cb59e093ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.508849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4bc11-0097-4298-96da-cb59e093ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.509326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4bc11-0097-4298-96da-cb59e093ecf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.522785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4bc11-0097-4298-96da-cb59e093ecf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3ac4bc11-0097-4298-96da-cb59e093ecf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:21.528628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:21.534550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 206/424] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:21.535335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.536133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.536133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.554576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:21.556914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:21.612122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cb347-f2f6-4569-b198-1becbdd7d65f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:21.637308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 207/425] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:21.638674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.657425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:21.658053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.658378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.658653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.660384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:21.662014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:21.664103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 219/426] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:21.669402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.830776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79284b-7a1f-4b3d-be21-a4a95c88e6ce None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:21.832480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:21.852697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:21.869894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaa93f-b012-47fb-88f7-07a4042e1f00 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:21.913092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:21.913719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.913994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.914258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.914936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:21.915235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:21.916698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:21.932364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:21.932364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:21.932364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:21.932364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:21.933500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 208/427] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:21.934999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:21.936473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:21.938041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 220/428] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:21.941740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:21.944160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.056602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3ed53-3575-42a7-badc-5d11dc0e0fd0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.077565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99605bf2-dd4d-4081-89ea-fa6ac654ef40 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.104101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a47390-6a3a-40b0-98f7-70c839447e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.105421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47390-6a3a-40b0-98f7-70c839447e56 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.106103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47390-6a3a-40b0-98f7-70c839447e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.106787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a47390-6a3a-40b0-98f7-70c839447e56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.140772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.142573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.143362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.144106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.145652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:22.146425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:22.149273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:22.164552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 221/429] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:22.164552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.190521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:22.201669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:22.250133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a47390-6a3a-40b0-98f7-70c839447e56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:22.251701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a47390-6a3a-40b0-98f7-70c839447e56 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:22.373407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e82915-c823-49fa-b201-2c08c1b651f2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.414659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 209/430] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:22.419584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5885a-fe8b-4e68-a655-14073350a24c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.420626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5885a-fe8b-4e68-a655-14073350a24c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.421199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5885a-fe8b-4e68-a655-14073350a24c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.422721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 210/431] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:22.423342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5885a-fe8b-4e68-a655-14073350a24c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.434935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9b8e7-a0fe-485e-9047-3716e6f6a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.436149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9b8e7-a0fe-485e-9047-3716e6f6a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.436149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9b8e7-a0fe-485e-9047-3716e6f6a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.436465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9b8e7-a0fe-485e-9047-3716e6f6a7a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.455493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9b8e7-a0fe-485e-9047-3716e6f6a7a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-9df9b8e7-a0fe-485e-9047-3716e6f6a7a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:22.465428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:22.466371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 16:25:22.467146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:22.470147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 222/432] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:22.471750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.472175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.473014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.499463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:22.500697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:22.526233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:22.562994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5885a-fe8b-4e68-a655-14073350a24c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.594965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 223/433] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:22.595904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.599753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:22.601934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cfc1c-2047-4a31-b9eb-4acf1b967894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.602587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cfc1c-2047-4a31-b9eb-4acf1b967894 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.603269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cfc1c-2047-4a31-b9eb-4acf1b967894 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.603889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cfc1c-2047-4a31-b9eb-4acf1b967894 None admin] PATH_INFO: `/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.606230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cfc1c-2047-4a31-b9eb-4acf1b967894 None admin] RBAC: Authorizing `identity:get_project(project_id=e50fa06fb1774ed1b1cc4471b8e86025)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:22.607787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cfc1c-2047-4a31-b9eb-4acf1b967894 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:22.611839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 211/434] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:22 2022] GET /identity/v3/projects/e50fa06fb1774ed1b1cc4471b8e86025 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:22.611839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbf8ed-9700-4ea0-bc80-64a008d8ef84 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.611839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbf8ed-9700-4ea0-bc80-64a008d8ef84 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.611839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbf8ed-9700-4ea0-bc80-64a008d8ef84 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.612488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 212/435] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:22.617093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbf8ed-9700-4ea0-bc80-64a008d8ef84 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.723279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883b2dd-685e-495e-98ef-56be24e3089e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.743692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbf8ed-9700-4ea0-bc80-64a008d8ef84 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.766264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.767209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.767702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.768144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.769211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:22.769561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:22.775073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9295e-8a85-4e8c-ae76-18cf5d360ad0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:22.785555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.786375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 224/436] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:25:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:22.788723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.789162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.789441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.790660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.791011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.791294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:22.792586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:22.794645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:22.801087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 213/437] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:22 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:22.801087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.813660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:22.817183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:22.818542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:22.890183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:22.897625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 225/438] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:22.898358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:22.913374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57b31a-d812-4b14-b90a-6fbaa986c564 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:22.951372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:22.951986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:22.952265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:22.952852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.002348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5755d-5b2c-40a0-a85c-e69307e55f3a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.026154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.028168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.037899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.038505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.038778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.039047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.039153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.039782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:23.040090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.041547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.059018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 226/439] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:22 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=e50fa06fb1774ed1b1cc4471b8e86025&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:23.059018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.124739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.178947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84b4ab-f2fb-48c4-94ab-3ff05fbf3ac0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.179865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 214/440] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:23.180843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.222797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.223452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.223729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.223998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.238436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:23.244531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:23.245450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:23.247076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 227/441] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:23.247736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.302621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199639c-69aa-482a-b92b-fda4193a4574 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.314556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.343790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.344476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.344758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.345031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.357398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:23.362252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:23.363281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 215/442] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:23.364211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.374547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8234fd-e270-4711-9b56-6ddfe3e92320 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.394388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.417689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.418721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.418897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.419167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.420334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.421832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.423493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 228/443] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:23.424329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.491643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea2d5-73ea-4f1e-be24-9be850cfacfb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.533224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298221c7-cfff-45cc-a756-2553daecaeb1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.533624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.534231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.534502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.534774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.535432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:23.535729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.537140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.545247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 216/444] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:23.545827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.570490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.571018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.571291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.571553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.572197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:23.572523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.573865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.576634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.582538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 229/445] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:23.583398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.631584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.644465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4817c-84ca-4119-8fd3-196a2e5ee073 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.677450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.678082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.678357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.678628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.679739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.681226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.681826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a6026-88c5-46a2-a125-184a6bcad572 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.682795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 217/446] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:23.684075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.716146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.716781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.717064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.717335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.777583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.781549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f6aec-ad47-4a0c-b34d-872c210f719f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:23.784799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.788473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.816361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:23.816993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:23.817273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:23.817551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:23.835786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.885299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:23.886843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:23.891777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 230/447] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:23.891777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.983384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:23.987394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 218/448] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:23.988184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:23.995549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc761133-b130-4bfa-8269-dff748ff3fe6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.031061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.031659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.031939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.032209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.032960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=af9a72cd6bb0442f9ebb70e3f609fb5c, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:24.033270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.034701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.040302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 231/449] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=af9a72cd6bb0442f9ebb70e3f609fb5c => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:24.041099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.058111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:24.111443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedcdb4-7f09-4f4d-9a32-7af90b7b1e6f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.154671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.155943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.156428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.156874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] PATH_INFO: `/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.158319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] RBAC: Authorizing `identity:get_project(project_id=f6b0410a3e194eb78e4f1dcc1cbe7c54)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.160009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.161383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d416e8-3731-46a6-81b6-2976a003925a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.162368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 219/450] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:23 2022] GET /identity/v3/projects/f6b0410a3e194eb78e4f1dcc1cbe7c54 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:24.167789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.208529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.208751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.208958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.209231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] PATH_INFO: `/v3/projects/86cc49d474614954959fd1fd2c69fbe7` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.212090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] RBAC: Authorizing `identity:get_project(project_id=86cc49d474614954959fd1fd2c69fbe7)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.213614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.215292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 232/451] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/projects/86cc49d474614954959fd1fd2c69fbe7 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:24.222236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.249604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:24.288794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36445df8-bb7d-49fa-aead-d1b77d1664bd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.317984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:24.334851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.335490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.335766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.336295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] PATH_INFO: `/v3/groups/a155663a0d334bd88ae938e0c2297a9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.338890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] RBAC: Authorizing `identity:get_group(group_id=a155663a0d334bd88ae938e0c2297a9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.341054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.342855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 220/452] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/groups/a155663a0d334bd88ae938e0c2297a9e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:24.343746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b646ee-d681-41dd-81c4-65534f562035 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.348588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.390933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.391522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.391799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.392070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] PATH_INFO: `/v3/groups/20e46fa44a6d4b0f920653feb35bb43e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.393311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] RBAC: Authorizing `identity:get_group(group_id=20e46fa44a6d4b0f920653feb35bb43e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.394753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.396404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 233/453] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/groups/20e46fa44a6d4b0f920653feb35bb43e => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:24.402085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:24.457938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb580-27e1-40fd-b1db-7d94c9a9b1a6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.490594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.491193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.491467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.491729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.492477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] RBAC: Adding query filter params (group.id=a155663a0d334bd88ae938e0c2297a9e, role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:24.492777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.494213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418e0aa-537a-407f-bbd7-c07f7a857d6f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.496504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:24.500944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 221/454] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/role_assignments?group.id=a155663a0d334bd88ae938e0c2297a9e&scope.project.id=f6b0410a3e194eb78e4f1dcc1cbe7c54&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:24.503946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383d394-2f30-45ba-b3ed-bb51e4987962 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:24.545984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:24.546601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:24.546877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:24.547147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:24.547887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] RBAC: Adding query filter params (group.id=20e46fa44a6d4b0f920653feb35bb43e, role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=86cc49d474614954959fd1fd2c69fbe7) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:24.548192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:24.549705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b203f-948c-4398-8338-3a877e0a7831 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:24.557445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 234/455] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Aug 24 16:25:24 2022] GET /identity/v3/role_assignments?group.id=20e46fa44a6d4b0f920653feb35bb43e&scope.project.id=86cc49d474614954959fd1fd2c69fbe7&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:26.939187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc5acd-ac5d-4461-a05b-b04e44f86169 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:26.939692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc5acd-ac5d-4461-a05b-b04e44f86169 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:26.939764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc5acd-ac5d-4461-a05b-b04e44f86169 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:26.941365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 222/456] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:26.947651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ace18-0b8f-4eb9-9fca-d4e600a2531e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:26.948455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ace18-0b8f-4eb9-9fca-d4e600a2531e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:26.949494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ace18-0b8f-4eb9-9fca-d4e600a2531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:26.977282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-392ace18-0b8f-4eb9-9fca-d4e600a2531e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:26.978454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-392ace18-0b8f-4eb9-9fca-d4e600a2531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:26.990413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:27.059276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 235/457] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:27.273953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937439b1-60a7-4ffe-85de-4fce546fdc82 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.274346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937439b1-60a7-4ffe-85de-4fce546fdc82 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.274526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937439b1-60a7-4ffe-85de-4fce546fdc82 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.276080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 223/458] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:27.281465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.282009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.282472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.309705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:27.311211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:27.380880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f364e0-fec2-44ac-beff-bda4018c490e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.381535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f364e0-fec2-44ac-beff-bda4018c490e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.381997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f364e0-fec2-44ac-beff-bda4018c490e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.383644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 224/459] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:27.384370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 236/460] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:27.389466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.390006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.390489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.421480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:27.423007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:27.483216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:27.514375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 225/461] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:27.571599 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:27.603068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f933f1-8a8e-455e-a027-eba73c09e783 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:27.638158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:27.638776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.639052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.639320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.649966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:27.651408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:27.666289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:27.667311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:27.668208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:27.668208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:27.668208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:27.668208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 237/462] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:27.673280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:27.674451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:27.781009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8be4e-f433-4269-9b6e-011413fb3701 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:27.811315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:27.824961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:27.824961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.824961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.824961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.833814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20162f3c-382a-4569-bc6a-bebc5a32d43e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:27.848291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:27.850247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:27.863046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ed3d6abb-e462-4927-84aa-833b17ef5fdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:27.874931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:27.875870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:27.876819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:27.876819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:27.876819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:27.878302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 226/463] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:27.882516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896589e-04cf-496b-b8d2-f85319a271a6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.882516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896589e-04cf-496b-b8d2-f85319a271a6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.882516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896589e-04cf-496b-b8d2-f85319a271a6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.887930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 227/464] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:27.887930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.887930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.888197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.888635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:27.889489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:27.889934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:27.890377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:27.891318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:27.891791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:27.896629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:27.911435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:27.911692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 238/465] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:27.912673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:27.912792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.000406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 228/466] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:28.000406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.027715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de796fa9-e003-4180-bb5a-4d59da2b745a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.067719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.068735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.069222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.069632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.070401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:28.070853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.072597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.090818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 239/467] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:28.091982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.115696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb44077-2b05-484f-92ab-d9f186296ddc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.127354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:28.142739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:28.153552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.154351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.154682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.155477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.165887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.167172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:28.184202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:28.189245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:28.190118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:28.190118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:28.190118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:28.190118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 229/468] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:28.190118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.203766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54d4da-28f4-409e-b7d2-3feba63eb8ee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.252037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.252688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.252965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.253236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.253877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.256033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.305198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac74e-9746-4836-a8fe-392cbbde472a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.342076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900784ef-dd44-418a-996b-49119986e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.342603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900784ef-dd44-418a-996b-49119986e96f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.342873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900784ef-dd44-418a-996b-49119986e96f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.343140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900784ef-dd44-418a-996b-49119986e96f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.343765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900784ef-dd44-418a-996b-49119986e96f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.345303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900784ef-dd44-418a-996b-49119986e96f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.356577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 240/469] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:28 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:28.356577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.423780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 230/470] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:28 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:28.425474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900784ef-dd44-418a-996b-49119986e96f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.453234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:28.517880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:28.523330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e94f7-533e-43fb-988e-fd124ca73223 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.569516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900784ef-dd44-418a-996b-49119986e96f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.606101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.606636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.606931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.607176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.607821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:28.608117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.609747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.614833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.615650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.616067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.616573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.627402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 241/471] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:28.629891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.632193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.634512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:28.652875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:28.654033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:28.655339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:28.655339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:28.655339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:28.657758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 231/472] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:28.662552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.756811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbef986-baa3-4254-961d-f24783c27a4d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.758973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:28.805976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aa645-1b6f-4290-a178-97fbe0e4ef99 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:28.806456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e783ebb-68e9-44f5-9916-e93c5874b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.807308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e783ebb-68e9-44f5-9916-e93c5874b525 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.807875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e783ebb-68e9-44f5-9916-e93c5874b525 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.808945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e783ebb-68e9-44f5-9916-e93c5874b525 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.810659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e783ebb-68e9-44f5-9916-e93c5874b525 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.812256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e783ebb-68e9-44f5-9916-e93c5874b525 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.853155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:28.853773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.854053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.854327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.855722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:28.855956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:28.862845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7a857-a663-43ee-8f5b-83ca41bd4f5b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:28.876071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 232/473] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:28.880370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893a1e-b1b4-42a9-9768-098e3a783624 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.880370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893a1e-b1b4-42a9-9768-098e3a783624 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.880370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9893a1e-b1b4-42a9-9768-098e3a783624 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.880370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 233/474] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:28.882642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9893a1e-b1b4-42a9-9768-098e3a783624 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:28.921719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 242/475] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:28 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:28.923284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:28.923834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:28.924118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:28.947912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:28.950020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:28.967305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:29.006718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9893a1e-b1b4-42a9-9768-098e3a783624 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:29.046405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 243/476] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:29.057843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:29.058628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.058914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.059193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.059868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:29.060382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:29.068685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:29.192615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 234/477] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:28 2022] POST /identity/v3/users => generated 249 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:29.439994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:29.544588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b72f-3660-4e59-975e-e95d470a879e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:29.585152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:29.585762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.586228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.586496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.587174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:29.588561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:29.653110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.sql.core [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:25:29.653110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 16:25:29.653110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [parameters: {'id': 'a16e04377c924f6d8c15ba97228c34ae', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 16:25:29.653110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81626) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 24 16:25:29.669829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:25:29.670762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application yield resource Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise self.value Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application session.commit() Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:25:29.671674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:25:29.672587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:25:29.673500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:25:29.674310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [parameters: {'id': 'a16e04377c924f6d8c15ba97228c34ae', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:29.675124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 16:25:29.675927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:29.678652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 244/478] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Aug 24 16:25:29 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:29.678797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:29.796158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5820d972-874b-4740-a730-c3370b3783d2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:29.838129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d4b90-4c56-44d6-8d88-dfe2386204d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:29.838763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4b90-4c56-44d6-8d88-dfe2386204d3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:29.839036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4b90-4c56-44d6-8d88-dfe2386204d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:29.839328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4b90-4c56-44d6-8d88-dfe2386204d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:29.859125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d4b90-4c56-44d6-8d88-dfe2386204d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1d5d4b90-4c56-44d6-8d88-dfe2386204d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:29.864372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:29.865258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:29.866034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 235/479] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:25:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:29.872056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:29.886898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:29.985186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d693e83-a5c7-427a-843d-d810d100c678 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:30.022150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:30.022758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.023032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.023298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.023969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:30.024255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:30.025651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68f6ba-a66c-4319-801b-9598eb22e4a9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:30.040555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 245/480] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:30.794022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d87e7-fcdb-45f6-904a-8ccdad393e8b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.794561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d87e7-fcdb-45f6-904a-8ccdad393e8b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.794638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d87e7-fcdb-45f6-904a-8ccdad393e8b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.796281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 236/481] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:30.801686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388eb83-411d-44aa-94f8-c7489384ea41 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.802092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388eb83-411d-44aa-94f8-c7489384ea41 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.802259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388eb83-411d-44aa-94f8-c7489384ea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.825346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f388eb83-411d-44aa-94f8-c7489384ea41 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:30.826530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f388eb83-411d-44aa-94f8-c7489384ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:30.906171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 246/482] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:30.960134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e6f1b-8638-4454-99e1-ce95e4dfb379 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.960452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e6f1b-8638-4454-99e1-ce95e4dfb379 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.960701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e6f1b-8638-4454-99e1-ce95e4dfb379 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.962025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 237/483] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:30.968158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524d5e0-8adc-4baf-8114-f123156a6f67 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:30.968463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524d5e0-8adc-4baf-8114-f123156a6f67 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:30.968738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524d5e0-8adc-4baf-8114-f123156a6f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:30.975081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:30.990387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e524d5e0-8adc-4baf-8114-f123156a6f67 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:30.991558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e524d5e0-8adc-4baf-8114-f123156a6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:31.063622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 247/484] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:31.205642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e6f1b-8638-4454-99e1-ce95e4dfb379 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:31.273465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:31.349692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e6f1b-8638-4454-99e1-ce95e4dfb379 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:31.394077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8f416-f99e-4099-ae73-05179faccb90 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.394684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8f416-f99e-4099-ae73-05179faccb90 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.395156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8f416-f99e-4099-ae73-05179faccb90 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.396468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:31.397201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.397403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.397598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 248/485] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:31.397692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.403542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.404077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.404596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.413532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:31.419574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:31.420756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:31.421555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 238/486] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:31.428137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:31.431894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:31.433390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:31.519128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 249/487] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:31.524611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:31.600217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43577aa-90c3-4652-b31c-00563771ec4e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:31.609118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:31.622919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:31.668464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:31.669073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.669343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.669608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.670278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:31.670574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:31.671967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:31.691453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 239/488] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:31.697468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:31.711444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dd869-5116-45dc-b773-db9407f3804b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:31.755878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:31.756538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.756817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.757091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.778411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:31.784648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:31.785553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:31.786348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 250/489] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:31.793392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:31.875800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de41edb-ba62-4694-bc68-eabf9b5d0fe5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:31.915370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:31.916007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.916286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.916589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.922760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cd78d-8942-4c9c-8057-ce24294fa9af None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:31.928149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:31.929695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:31.944955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:31.945827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:31.946652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:31.946652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:31.946652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:31.946652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 240/490] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:31.946858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:31.955417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:31.967983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:31.968620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.968897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.969166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.978272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:31.978272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:31.978272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9aef45-d2d6-4940-ad68-89a525157b68 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:31.978532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:31.985673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 251/491] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:31.985673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fb4d-1d59-46ab-b922-a9409aa65f36 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:31.985673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fb4d-1d59-46ab-b922-a9409aa65f36 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:31.985673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50fb4d-1d59-46ab-b922-a9409aa65f36 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:31.985673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 252/492] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:31.985673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50fb4d-1d59-46ab-b922-a9409aa65f36 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.068747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f25b43-5245-46ba-aca4-95c92addc842 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.094119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50fb4d-1d59-46ab-b922-a9409aa65f36 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.105528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.106235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.106531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.106808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.117857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:32.123681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:32.124596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:32.125433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 241/493] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:32.126118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.134356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.134975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.135275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.135536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.136815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:32.137130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.138531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17f912-76e7-4845-8524-4dc46906178b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.150423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 253/494] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:32.151694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.152595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.152877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.171748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:32.172974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:32.215293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.221808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.241986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e6da1-f199-4ab9-a932-3a0fcbcfac4a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.242183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 254/495] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:32.242840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.295542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.296805 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.297260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.298013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.313367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.315596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:32.333629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:32.334570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:32.336066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:32.336066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:32.336066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:32.336066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 242/496] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:32.336066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.369174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd242e9b-29f6-4c7c-b4e9-830feeb63077 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.411867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.413057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.413340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.413614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.414292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:32.414596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.415987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.430180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 255/497] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:32.431062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.452613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c775d-0eff-44d6-96d2-9be4841b446e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.469057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.484712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.493030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.493924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.494634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.495121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.505848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.508090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:32.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:32.525711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:25:32.526594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:32.534037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 243/498] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:32.534037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.546348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f304-fa89-48d3-b5f2-8da16692cb9f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.582582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.583228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.583506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.583774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.584440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:32.584852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.586334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.603462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 256/499] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:32.603462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.638101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ad9d-9a4d-417d-b768-8685a8c569b8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.674111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.674663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.674935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.675198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.685707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.687198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.692759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.698393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d57c1c-6ff7-446e-bd63-a51133e11704 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:32.701573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:32.702502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:32.703340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:32.703340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:32.703340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:32.703340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 244/500] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:32.703564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.734819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d2365-36fd-4ef1-a37d-ab21bf04e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.735439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d2365-36fd-4ef1-a37d-ab21bf04e903 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.735714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d2365-36fd-4ef1-a37d-ab21bf04e903 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.735977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d2365-36fd-4ef1-a37d-ab21bf04e903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.746312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d2365-36fd-4ef1-a37d-ab21bf04e903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6b8d2365-36fd-4ef1-a37d-ab21bf04e903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:32.751182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:32.752197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:32.753019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 257/501] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:32.754107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9b0a6-98da-4eee-9f5d-44bf9ec15d80 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.754107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9b0a6-98da-4eee-9f5d-44bf9ec15d80 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.754107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9b0a6-98da-4eee-9f5d-44bf9ec15d80 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.754733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.755291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 258/502] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:32.755775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9b0a6-98da-4eee-9f5d-44bf9ec15d80 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.809466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad868038-4b2c-49d7-9067-234f3d7626c9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.842688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.843349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.843620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.843886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.844646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:32.844947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.846384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.867224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9b0a6-98da-4eee-9f5d-44bf9ec15d80 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.882977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 245/503] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:32.884256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.912070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:32.912736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:32.913012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:32.913275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:32.920429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:32.921864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:32.934593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:32.936090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 259/504] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:32.937211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:32.955872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:32.993322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba963b-0d4d-4da9-b41b-45611f048ac1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:32.994891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.030649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.031278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.031804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.032299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.033464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:33.033867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.035293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0140-ba3e-4e05-baea-23f677cd3015 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.042417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676537e-4d56-481d-ac10-e92fae245cee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.046770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 246/505] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:33.048096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.048413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.048684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.066482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:33.067691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:33.077762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.078360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.078641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.078907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.079555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:33.079853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.081243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.089742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 260/506] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:33.090538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.135568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 247/507] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:33.137224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.172281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.195914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52cf94-51e1-4e33-bfdf-15108fd2bae1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.226834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.233460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.234062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.234345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.234618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.235797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74723e-733a-474a-aeef-eb0111fe4faa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.274026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.274637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.274912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.275183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.275908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:33.276211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.277743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.302734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 248/508] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:33.303581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.323244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.325493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.410482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.422670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860203c-4daf-433b-beef-ecbfddffd890 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.431974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 261/509] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:33.433360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.457587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.458139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.458413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.458739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.465839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.467298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:33.478659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:33.479633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 249/510] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:33.480648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.492271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.537955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22cda5-f1ba-4fbc-86ce-0b865f2aba22 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.571778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.572422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.572697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.572967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.581332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367499ee-28c9-4128-a399-a210bbe09b91 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.582628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.584078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:33.598711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:33.599620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:33.600503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:33.600503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.600503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:33.600503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 262/511] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:33.600756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.618784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.619385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.619662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.619928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.631492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.690966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.692480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.705637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.713898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e54c7a-4443-44ad-8bb3-e3058ac3709d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.747932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.748595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.748872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.749138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.758882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.760305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:33.776544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:33.778078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:33.778930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:33.778930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.778930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:33.778930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 263/512] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:33.778930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.796972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 250/513] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:33.797804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.864035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.882503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ab979-3faa-441c-9bb1-99ba65d3763c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.899161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0c8e8-fe4e-434e-a9ae-d6b35dba942c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:33.911310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:33.917457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.918081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.918359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.918631 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.928660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.930093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.934642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:33.935250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:33.935529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:33.935841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:33.936611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:33.936911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:33.938380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:33.944972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:33.946190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:33.947037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:33.947037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:33.947037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:33.947037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 264/514] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:33.947037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:33.957452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 251/515] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:33.959456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.050389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c62f1b-3f34-43e6-87e9-ee3e269176b8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.064871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f866-4526-4974-8bd5-876b4ea3530c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.085562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.086163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.086447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.086720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.087362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:34.087667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.089202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.090651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.099589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.100194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.100450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 265/516] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.101246 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.101362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.101615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.102276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:34.102585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.104058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.115783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 252/517] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.116707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.145265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.202813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7ef5-59dc-444d-977f-a682def3923f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.218255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28769109-1d9e-4972-afe5-9cb21afabe27 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.235502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.236127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.236438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.236715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.247659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.249209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.250667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.251332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.251609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.251877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.252543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:34.252842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.254204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:34.263587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:34.264679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:34.265583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:34.265583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:34.265583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:34.265583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 266/518] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:34.265889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.271052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 253/519] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.271052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.315256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.354534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.366827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79cab-91eb-4e4d-ac28-0714c032848f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.367646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c5446-7203-4fe4-b039-13c1cf95f323 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.400954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.401166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.401660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.401767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.401984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.402093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.402235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.402324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.409379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.410834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:34.422682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:34.423705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 254/520] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:34.424826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.470099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.471546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.516466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.526955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a812fc7-6758-41b9-8237-8baec914f46d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.560586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.561198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.561470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.561734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.562380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.563838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.569797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 267/521] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:34.570569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.641171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 255/522] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:34 2022] POST /identity/v3/users => generated 247 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:34.641978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.645565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.681193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72421-59a9-43fe-8cba-d1e2a70ac17c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.719724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.720509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.720857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.721198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.721891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:34.722180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.723608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.736873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 268/523] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.737738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.748901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.764117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95435e-2354-42bd-ac8d-c3a129b7cbea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.804018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.804702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.804979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.805247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.815253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.816773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:34.830790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:34.831651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:34.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 256/524] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:34.833554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.864945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ece529-f962-403c-b580-73cd2a86569e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.907178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.907863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.908206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.908331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:34.908777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:34.909424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:34.909798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:34.911345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:34.950580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 269/525] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:34.952279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:34.958907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e798b-9e68-4ed9-b111-a55fdcf5f124 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:34.997359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:34.998134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:34.998475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:34.998803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.009986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.011693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.018901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:35.026350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:35.027976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:35.028936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:35.028936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:35.028936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:35.032607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 257/526] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:35.034598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.062553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773583c-83e7-4471-b6b6-69739dc9874e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.105863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.105863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.105863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.105863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.107581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.109068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:35.121124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:35.122025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:35.122861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 270/527] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:35.123334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.148115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9200-636e-4865-82f5-6a83200aab84 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.165488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.186564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.187342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.187619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.188231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.188957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:35.189263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.190716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.210325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 258/528] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:35.212004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.243719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bcb545-ca38-4550-b592-c59dbaa74c98 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.270156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.284055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.285092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.285644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.285922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.331539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3323d6-6bc8-40bd-a451-39d2618961d9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.361659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.363061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.367606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.368210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.368512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.368781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.369421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:35.369717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.371902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.387833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 259/529] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:35.387833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.423590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.473038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 271/530] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:35.474208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.499332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5195c30-0908-46cc-b9ad-2f093ea87d8d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.500708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.535386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.536093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.536395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.536667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.537360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:35.537664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.539104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.559522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 260/531] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:35.559522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.579600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b29f0-adc0-4963-b607-f5e48a62a5a6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.615582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.616218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.616504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.616776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.617496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=43cdc8de558d4087bed8a3bb48cf4bfd) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:35.617796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.619439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.636633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 272/532] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/role_assignments?user.id=43cdc8de558d4087bed8a3bb48cf4bfd&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:35.636633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.680539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.689007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4a3d7-2328-4858-8df6-6e06485b957c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.739902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.741399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:35.742260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.742550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.743160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.753323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.754556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a28e28-a711-48f2-b35a-70ac4fefec10 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.754833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:35.770952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:35.771886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:35.772756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:35.774376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 261/533] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:35.774376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.801638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.802433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.802804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.803174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.815328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.816952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:35.836012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:35.836985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:35.837997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:35.837997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:35.837997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:35.837997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 273/534] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:35.839693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:35.922863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b154a-996b-4116-87e8-b76201cb35e9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.951054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3732d-08c5-4af6-b203-832066438298 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:35.971628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.972639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.973191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.973604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.974638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=4542cf167cc24738b441af6530dead3c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:35.975428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.977633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.992375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:35.992375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:35.992375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:35.992375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:35.992801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:35.992801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:35.992801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:35.993615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 262/535] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/role_assignments?user.id=4542cf167cc24738b441af6530dead3c&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:35.994853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:36.022489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:36.028933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:36.035732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 274/536] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:36.041482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:36.128079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5937e-aedf-4ce0-a430-edf8697336e4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:36.175013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:36.176068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.176993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.177471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.178343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:36.178852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:36.181278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:36.197511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 263/537] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:36.204985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:36.272503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58012a3-2d49-4bf9-828b-d337f4060ecd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:36.310058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:36.318910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:36.319637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.319914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.320180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.321294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=ef06aba40382479289b1e9e19395a2be) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:36.321294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:36.322704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:36.330282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 275/538] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:36 2022] GET /identity/v3/role_assignments?user.id=ef06aba40382479289b1e9e19395a2be&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:36.338399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b81ca-5158-4ad2-a242-cac5172f6321 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:36.380480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:36.381132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.381405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.381671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.389926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:36.391414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:36.404869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:36.405776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:36.406773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 264/539] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:36.413460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:36.486795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:36.528967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbc502-a1ec-4848-bc18-473015c5204f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:36.569381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:36.570421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.570906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.571350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.572216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:36.572707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:36.574927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c599f9-6b4d-4d61-ac9a-40b321490d3b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:36.592445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 276/540] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:36.594875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:36.633795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb99983-b967-4f5b-9614-06a3069ff81d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.634391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb99983-b967-4f5b-9614-06a3069ff81d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.634889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb99983-b967-4f5b-9614-06a3069ff81d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.637123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 277/541] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:36.642388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.642665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.642933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.663758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:36.665101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:36.753303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b02d79-ba9e-4c5f-aaeb-4a5ca0800f32 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:36.766547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 278/542] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:36.815295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:36.815966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:36.816244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:36.816537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:36.817268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=9bde9ceb3c7c46298626580b62e0ec01) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:36.817572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:36.819136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:36.830014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 265/543] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:36 2022] GET /identity/v3/role_assignments?user.id=9bde9ceb3c7c46298626580b62e0ec01&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:37.131132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.210340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.272928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24868f-9bbb-4c42-bdd0-3681a62038b4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.322369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.324151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.324909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.325392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.340477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:37.345644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:37.346534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:37.347346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:37.350030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 279/544] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:37.356195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.381467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.541090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcda08-817e-4c95-b833-07b16165db15 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.588035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.593387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.593675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.593951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.594659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:37.594964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:37.596401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:37.610036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 266/545] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:37.617465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.789362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f5dd4-5c08-47c0-ac82-ed8ed16f93af None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:37.812365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:37.838484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:37.839657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:37.839932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:37.840195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:37.860053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:37.861537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4b865fe6-fef9-49fd-afa0-5fa58e5b8f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:37.888089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:37.889077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:37.889953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:37.889953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:37.889953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:37.889953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 280/546] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:37.898280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:37.992846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:38.040911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda71cea-67ab-4c28-a819-b5ec5a7441a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.061095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77479cd-d15d-49d9-ba65-b995dff27288 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.061447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77479cd-d15d-49d9-ba65-b995dff27288 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.061633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77479cd-d15d-49d9-ba65-b995dff27288 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.063309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 281/547] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:38.074657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.074940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.075211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.092255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.092892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.093167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.093439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.094143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:38.094659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.095830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.111083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 267/548] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:38.117515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.122307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:38.123494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:38.258815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775780d-5ee0-4662-8b96-f790683b5a69 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.273832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 282/549] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:38.287464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:38.302692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.302692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.302692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.302692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.305149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.306628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:38.318926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:38.321209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:38.324045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 268/550] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:38.328940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.457008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:38.463023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b199ef4-0561-45e5-845a-21d7bbeb621b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.502941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.503558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.503832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.504098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.504860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:38.505158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.506583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.524944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 283/551] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Aug 24 16:25:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:38.531086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.652813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602bcd0-956c-4d79-a1b2-7e8c23376e16 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.695980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154859e-bc79-4d35-acfd-0e001e12efc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.696627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154859e-bc79-4d35-acfd-0e001e12efc7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.696913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154859e-bc79-4d35-acfd-0e001e12efc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.697151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154859e-bc79-4d35-acfd-0e001e12efc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.757154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:38.771798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154859e-bc79-4d35-acfd-0e001e12efc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:38.779822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154859e-bc79-4d35-acfd-0e001e12efc7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:38.789584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.901980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 269/552] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:38.901980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511d238-ba5e-4d87-b52b-fa3c2cbf674e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.901980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511d238-ba5e-4d87-b52b-fa3c2cbf674e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.901980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511d238-ba5e-4d87-b52b-fa3c2cbf674e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.903581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 270/553] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:38.905128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511d238-ba5e-4d87-b52b-fa3c2cbf674e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:38.942336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e1bf6-6777-4267-a31d-6b47c905ecff None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:38.981192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01541b37-931e-4431-a377-2916e4293ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:38.981828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01541b37-931e-4431-a377-2916e4293ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:38.982194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01541b37-931e-4431-a377-2916e4293ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:38.982362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01541b37-931e-4431-a377-2916e4293ec6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:38.993327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01541b37-931e-4431-a377-2916e4293ec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-01541b37-931e-4431-a377-2916e4293ec6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:38.998511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:38.999640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:39.000862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 284/554] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:39.001661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.001661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.001661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.014694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.021061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:39.022323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:39.034187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.035740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511d238-ba5e-4d87-b52b-fa3c2cbf674e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.079921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.080925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.081410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.081862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.094388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.096287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.098570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 285/555] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:39.098570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9265ba3d-38ae-44ff-9703-728c59676db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:39.133537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:39.134511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:39.135373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:39.135373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:39.135373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:39.138434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 271/556] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:39.140039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4992c-3645-427f-8a8f-163a28d15fd8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.140622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4992c-3645-427f-8a8f-163a28d15fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.141066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4992c-3645-427f-8a8f-163a28d15fd8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.142904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 272/557] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:39.143842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4992c-3645-427f-8a8f-163a28d15fd8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.214147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863c356-d9f0-4253-92f9-b1651be264b9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.252852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.253547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.254131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.254233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.255027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:39.255333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.256867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b770f-bc99-4a37-8b66-f48a57648a3a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.266797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 286/558] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:39.268490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03a32-e5e8-467f-ae46-660b97669449 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.268601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03a32-e5e8-467f-ae46-660b97669449 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.268885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03a32-e5e8-467f-ae46-660b97669449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.277309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4992c-3645-427f-8a8f-163a28d15fd8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.283293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.288735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dad03a32-e5e8-467f-ae46-660b97669449 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:39.289933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dad03a32-e5e8-467f-ae46-660b97669449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:39.316274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.317402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.317876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.318340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.319172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:39.319878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.322481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.330466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.337202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 273/559] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:39.338215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.363561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 287/560] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:39.364945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77419d8c-30ab-4819-9ee2-e174cb481d3b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.365226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77419d8c-30ab-4819-9ee2-e174cb481d3b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.365497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77419d8c-30ab-4819-9ee2-e174cb481d3b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.368462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 288/561] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:39.368462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77419d8c-30ab-4819-9ee2-e174cb481d3b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.467470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e7d65-2ed8-4d30-876a-961073021864 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.487668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77419d8c-30ab-4819-9ee2-e174cb481d3b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.502769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.503349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.503631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.503902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.514564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.516120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.523777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.524444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.525121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.525396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.530116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3a75d917-098c-4370-9b5e-c2f4f38d6b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:39.531992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:39.533292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:39.534163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:39.534163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:39.534163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:39.534163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 274/562] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:39.534810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.534973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.535225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.544000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.546303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.557081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:39.558903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:39.561642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:39.562526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:39.563694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 289/563] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:39.564178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.615467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.637355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 275/564] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:39.638189 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.677291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34d184-a9b9-4968-811e-f6fcdc3e82ee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.715022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.715650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.715927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.716195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.728278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:39.733796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:39.734804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 290/565] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:39.735650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.749228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ff74-a81d-4d9b-a40c-6d88f8b3c96b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.785862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.786497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.786757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.787033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.787684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:39.788028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.789481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.798509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.802693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 276/566] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:39.803602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.848618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:39.849182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b139c3-3e08-4bb3-8e74-de317d9e4d2b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.886851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.887817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.888298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.888833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.889602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:39.889978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:39.891505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:39.905429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 291/567] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:39.906263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:39.915614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f486b-13a4-479d-8e03-53cbed09dd40 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:39.950347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:39.950969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:39.951247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:39.951515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:39.961314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:39.966378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:39.967428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 277/568] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:39.968238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.006278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba80f87-6e69-4a4a-8b58-2d403b38acc0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.022559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.040166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.040799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.041076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.041343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.042025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:40.042325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.043677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.051832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 292/569] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.052563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.065345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.068527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87de6a1-f91e-4d1c-ba74-466483675306 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.101319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.101935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.102209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.102474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.109983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.111518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:40.123813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:40.129491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 278/570] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:40.130659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.185669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9a799-d14a-4e64-8983-7bab0496d49d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.218770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.219389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.219663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.219936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.229771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:40.234792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:40.235873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:40.236773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 293/571] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:40.237453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.252100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712c94f-6fa8-4695-aa13-1771543cfbf8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.278629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.287931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.288576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.290817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.290817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.290817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=1f762db26df84f5faf7e6778383e2870) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:40.290817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.292144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.297840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 279/572] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/role_assignments?user.id=1f762db26df84f5faf7e6778383e2870&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.298687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.330319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.351706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa13c-666b-40da-aaae-afd4c9afbc83 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.393473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.394426 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.394888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.395171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.395838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:40.396172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.397712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.407900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 294/573] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.413438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.420007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955880a6-090f-478d-a502-13bc094d7dfd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.461100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.461831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.462109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.462382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.474940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.476693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:40.492883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:40.493858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:40.494785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:40.494785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:40.494785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:40.496659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 280/574] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:40.498241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.519353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.522807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72ed44-f51f-4968-92a7-f823a89fee7c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.560362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.560924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.561197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.561466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.562282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:40.562489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.563901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.584553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.590748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 295/575] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.591711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.617909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1abf3-d076-443e-bab0-7de6ad77fcf2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.663130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.663987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.664443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.664716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.665377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:40.665675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.667099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.680469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 281/576] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.680469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.709674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1878c-216c-40c7-ab86-bc8ca71c9337 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.749432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.750304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.750585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.751440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.764308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.765791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:40.780927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:40.786647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:40.787802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:40.787802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:40.787802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:40.787802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 296/577] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:40.787802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.794743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ec2a4-b5dd-49ed-a28a-897f93da6c56 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.800306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.830844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.831477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.831753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.832016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.832679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:40.832971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.834348 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.841991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:40.851245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 282/578] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:40.851245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:40.905300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff0357-5ac3-45cc-a4a4-a07f0bce5675 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.958078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:40.958733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:40.958847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cf722-1c4e-4a9e-8c55-236e0e620deb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:40.959009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:40.959275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:40.971464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:40.972974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:40.987870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:40.989013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:40.990084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:40.990084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:40.990084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:40.993706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 297/579] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:40.994316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.000073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.000855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.001233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.001602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.013322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.014909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:41.030506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:41.031609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:41.032697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:41.032697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:41.032697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:41.033392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 283/580] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:41.038482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.061640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.091744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.101775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754478e-e9b2-44d7-b3de-62d07f32d4ba None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.138215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.138816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.139092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.139356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.139987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:41.140282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.141660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.145344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb7345-602e-42bc-8597-2566bb7dcde8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.158757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 298/581] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:41.158757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.230201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.230928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.231266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.231595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.240043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.241630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:41.253730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:41.254842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:41.255991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:41.256903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 284/582] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:41.257493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.305461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14d4b7-9cdf-4f0d-a215-4718a98fef73 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.330870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.342707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.343243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.343518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.343785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.344448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:41.344866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.346224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.358791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 299/583] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:41.359722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.369086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201377f-8cab-4c05-b871-e22bdf5828a7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.381172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.405082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.405822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.406064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.406337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.407003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:41.407306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.408799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.426802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 285/584] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:41.426802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.472482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4c050-5bb6-4602-b8a5-5d48878cd75e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.506931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.507551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.507830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.508101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.515636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.517094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.528220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d349b-c73c-4eb9-806f-ef1f6a71ce1f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:41.528711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:41.529906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:41.530865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 300/585] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:41.531457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.568958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.569565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.569842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.570116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.570902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:41.571211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.572697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.576305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.591603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 286/586] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:41.592640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.624381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.642021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42de08a-1841-4e53-893f-8453c41746df None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.684228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.685090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.685265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.685502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.692712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.694124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.701751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf566da0-ee06-4e2f-adf9-870fd498ed08 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:41.705980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:41.706833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:41.707737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:41.711727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 301/587] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:41.712451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.737666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.738306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.738580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.738851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.747279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.748801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:41.761013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:41.761959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:41.762902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 287/588] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:41.763473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.827171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d2e41-725f-460f-ba8f-68f6ba6ed507 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.828849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.860057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:41.862938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.863531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.863954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.864060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.864834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:41.865111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.866530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.874066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a3abed-9779-4880-9be7-c95859d09f90 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:41.890216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 302/589] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:41.893295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:41.913808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:41.914707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:41.914995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:41.915273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:41.927856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:41.929399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:41.945458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:41.946431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:41.947290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:41.947290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:41.947290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:41.947290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 288/590] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:41.947524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.006374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb641223-ab91-42d5-aae6-7c54ffd45bbe None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.041208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.042000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.042280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.042551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.043237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:42.043541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.045172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.061226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 303/591] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:42.061226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.061466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827bf99-d8f6-48de-9602-a179665ecf3d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.083881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.098652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.105830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.105830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.105830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.105830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:42.105830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.105830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.112111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.124929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 289/592] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Aug 24 16:25:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:42.124929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.179648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086c65b-0f0f-4d44-968f-b97314ef1d54 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.217508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.218146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.218433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.218704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.229086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.229683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2f8de-4237-4ad3-88d3-c72fb783ffa3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.231009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:42.246491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:42.247617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:42.248937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:42.248937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:42.248937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:42.248937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 304/593] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:42.248937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.274800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f50540-3598-493f-8bde-67825a440cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.275400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f50540-3598-493f-8bde-67825a440cec None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.275702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f50540-3598-493f-8bde-67825a440cec None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.276075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f50540-3598-493f-8bde-67825a440cec None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.276907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f50540-3598-493f-8bde-67825a440cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:42.277208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f50540-3598-493f-8bde-67825a440cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.278586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f50540-3598-493f-8bde-67825a440cec None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.289916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 290/594] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:42.290808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f50540-3598-493f-8bde-67825a440cec None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.329478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.349052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.358914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c64ed-27d4-48ce-832f-e78c16a17f33 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.392819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f50540-3598-493f-8bde-67825a440cec None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.393572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472bfe-48bb-4b9d-8b4e-1e9d0113690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.394276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472bfe-48bb-4b9d-8b4e-1e9d0113690c None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.394583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472bfe-48bb-4b9d-8b4e-1e9d0113690c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.394838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472bfe-48bb-4b9d-8b4e-1e9d0113690c None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/43cdc8de558d4087bed8a3bb48cf4bfd/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.409822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4472bfe-48bb-4b9d-8b4e-1e9d0113690c None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=43cdc8de558d4087bed8a3bb48cf4bfd, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.411289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4472bfe-48bb-4b9d-8b4e-1e9d0113690c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.425874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.432784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.432784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.432784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.437854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.439359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:42.453777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:42.454871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:42.455933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:42.455933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:42.455933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:42.455933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 291/595] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:42.455933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.486820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 305/596] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:42 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/43cdc8de558d4087bed8a3bb48cf4bfd/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:42.488397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d5772-f426-4f5e-ba10-b86d52b63d15 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.488870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d5772-f426-4f5e-ba10-b86d52b63d15 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.489221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d5772-f426-4f5e-ba10-b86d52b63d15 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.490715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 306/597] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:42.496883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d5772-f426-4f5e-ba10-b86d52b63d15 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.566672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cfbf8-339f-4df0-abd0-2652c4b1164f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.578450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.605350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.606055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.606251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.606525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.607281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:42.607820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.609871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.615603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d5772-f426-4f5e-ba10-b86d52b63d15 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.627033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 292/598] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:42.628217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.646430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.661546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.662608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.662926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.663479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.671262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.672984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ab357974-e7df-44bd-8dc8-b53387848aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:42.686849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:42.687944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:42.689143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:42.694452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 307/599] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:42.694452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.694452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.694452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.711101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:42.712306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:42.740767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4d20-d171-46cf-bef8-c611e5353e42 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.778364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.778972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.779554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.779835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.780573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:42.780876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.788522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 308/600] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:42.788522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.788765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.801315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 293/601] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:42.802166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.835570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.912439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00eeaf-2f16-4736-9fd8-a23ffea4cfb0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.923945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6424f8-bfe6-47dc-b363-a17b350762ab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:42.927671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:42.952702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e596-023e-495b-96b6-2966187245fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.953500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e596-023e-495b-96b6-2966187245fe None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.954198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e596-023e-495b-96b6-2966187245fe None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.954408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288e596-023e-495b-96b6-2966187245fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.962120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:42.962752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288e596-023e-495b-96b6-2966187245fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.963091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:42.963697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:42.964127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:42.964756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288e596-023e-495b-96b6-2966187245fe None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.965296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:42.965833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:42.967426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c288e596-023e-495b-96b6-2966187245fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:42.977486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:42.978558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:42.979819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:42.981867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 309/602] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:42.983049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e596-023e-495b-96b6-2966187245fe None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:42.988821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 294/603] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:42.989905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.085440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:43.097046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dd240-4a04-4017-b100-a96cd543e23d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.098598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288e596-023e-495b-96b6-2966187245fe None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.133839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.134443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.134724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.134985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.136401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:43.136709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.137560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.138161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.138262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.138451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.138718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.146288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.147710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.155305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 295/604] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:43.155305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:43.165099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:43.166177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 310/605] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:43.167539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.202553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:43.265072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194d711-16b9-41a5-bfdc-4b95b4bc5558 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.279617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5d0d8-cccf-4f19-8ee2-9ed94ad5af18 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.301721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.302385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.302664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.302937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/4542cf167cc24738b441af6530dead3c/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.315180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.316105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.316351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.317742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=4542cf167cc24738b441af6530dead3c, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.319222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.327669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:43.332431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:43.336070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 311/606] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:43.337239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.386401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:43.390911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 296/607] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:43 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/4542cf167cc24738b441af6530dead3c/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:43.397387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.446164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:43.459048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7024d-3a55-49d7-bb67-d5a319751e25 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.509789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.511097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.511540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.511994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/ef06aba40382479289b1e9e19395a2be/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.535491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=ef06aba40382479289b1e9e19395a2be, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.537461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.538612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8677af-f386-4780-816d-07866a52df63 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.577870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.578494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.578767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.579041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:43.580073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.581676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.595080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 297/608] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:43.595936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.645192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 312/609] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:43 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/ef06aba40382479289b1e9e19395a2be/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:43.647125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.661849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:43.727562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42700758-990d-4b40-9ec1-0bd33221aef3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.769888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.770498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.770774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.771045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.771719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:43.772020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.773550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.786465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 298/610] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:43.788688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242fc3c-28ce-47bd-9d01-7e4bd9794488 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.793467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:43.834144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:43.835096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:43.835524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:43.836807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/9bde9ceb3c7c46298626580b62e0ec01/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:43.853565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:43.858007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=9bde9ceb3c7c46298626580b62e0ec01, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:43.860069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:43.917417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 313/611] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:43 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/9bde9ceb3c7c46298626580b62e0ec01/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:25:43.961541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31bb0c-51da-44e6-9fc9-8cd66f62d8e7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:43.992090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.006159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.006814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.007092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.007414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.020191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.021862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:44.038839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:44.039815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:44.040754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:44.040754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:44.040754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:44.040754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 299/612] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:44.047903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.181813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a06487-7466-42fb-ac8c-7c4a445a0303 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.218831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.219564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.220048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.220566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.221465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:44.221992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.224318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.247607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 314/613] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:44.255277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.405297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdc873-7bf9-4c91-b063-158ad63b87ac None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.457079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.457706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.457977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.458242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.470360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.471819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fea78bb5-6217-42a7-a21b-8bbe9fbd5a02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:44.487436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:44.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:44.489382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 300/614] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:44.497842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.576485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.596909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2df10-b16c-4691-9fe6-31ca7805c466 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.597191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2df10-b16c-4691-9fe6-31ca7805c466 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.597452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2df10-b16c-4691-9fe6-31ca7805c466 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.598770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 301/615] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:44.613797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.614163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.614541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.632064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9851f5e-493c-4aec-b1ec-be842fb14f86 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.651288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:44.652715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:44.679262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.679808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.680091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.680384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.681129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:44.681434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.683094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.704687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.727347 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 315/616] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:44.733454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:44.776594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 302/617] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:44.877862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8ee49-1e30-4196-8968-01c3f10720b0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:44.896120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:44.931369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:44.931968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:44.932273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:44.932571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:44.949181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:44.950681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:44.975735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:44.976757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:44.977649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:44.977649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:44.977649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:44.977649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 316/618] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:44.982531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.089379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc781d48-01d7-4f02-aab8-dc57842262c3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.126008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.126621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.126898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.127168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.127821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:45.128121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.129545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.142835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 303/619] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.147992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.164873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.222697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.277234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d4f9a-a3d9-4198-a55a-b44b42f0a3a2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.323358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.324415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.324897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.325693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.333200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.334879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:45.347276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:45.348199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:45.349154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:45.356466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 317/620] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:45.358143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.387628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.468567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c4cc9-b9b7-4ec9-9153-aa7aacd2bcb7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.482373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99138ea8-6122-4b98-9182-3e5414a50ca4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.507000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.508502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.509000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.509487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.511090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:45.511684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.513647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.531825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 318/621] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.533472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.617059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.617705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.617982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.618247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.624592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.645066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4197c94e-ba4a-46aa-882f-f697d5cc5549 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.657311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:45.662504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:45.671715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:45.673107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 304/622] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:45.685485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.687200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81b8a1-92f3-4bc7-a3a5-98d9e053aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.688041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b8a1-92f3-4bc7-a3a5-98d9e053aca6 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.688323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b8a1-92f3-4bc7-a3a5-98d9e053aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.688841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b8a1-92f3-4bc7-a3a5-98d9e053aca6 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/1f762db26df84f5faf7e6778383e2870/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.707569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b8a1-92f3-4bc7-a3a5-98d9e053aca6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=1f762db26df84f5faf7e6778383e2870, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.709067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b8a1-92f3-4bc7-a3a5-98d9e053aca6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.794317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:45.811191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1617ec-882e-49c6-8ecd-94df9b6720e1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:45.829258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 319/623] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:25:45 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/1f762db26df84f5faf7e6778383e2870/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:25:45.835135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106299f4-967d-4b17-b62d-42daf218c4e5 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.835421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106299f4-967d-4b17-b62d-42daf218c4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.835744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106299f4-967d-4b17-b62d-42daf218c4e5 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.839769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 320/624] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:45.844942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5d93-316d-4a21-93a5-4ff7ed478cf5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.845284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5d93-316d-4a21-93a5-4ff7ed478cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.845626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e5d93-316d-4a21-93a5-4ff7ed478cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.855497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:45.856232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:45.856584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:45.856862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:45.857547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:45.857845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:45.859535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:45.871584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 305/625] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:45.882133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:45.886451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-df5e5d93-316d-4a21-93a5-4ff7ed478cf5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:45.887477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-df5e5d93-316d-4a21-93a5-4ff7ed478cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:46.019788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 321/626] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:46.072558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4284-5289-4caa-83e0-1388c12b9a67 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.111358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.112497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.112998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.113523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.125944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.127745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.130284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81512-29c7-46a6-b219-03a2cc6c1b17 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.130582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81512-29c7-46a6-b219-03a2cc6c1b17 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.130857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81512-29c7-46a6-b219-03a2cc6c1b17 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.132971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 322/627] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:46.140514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.140634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.140634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.140634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:46.147407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:46.148460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:46.149363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:46.149363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:46.149363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:46.156490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 306/628] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:46.158638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.177995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:46.182594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:46.297237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.317706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 323/629] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:46.336284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.352833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264897-e854-49bc-8ba4-ec6bcffdf7a9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.393869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.394762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.395193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.395716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.396541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:46.397132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.398686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aa0b8-16db-4171-9037-84e0d6c2e55e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.414501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 307/630] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:46.415976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9411f-8af0-4fba-b91f-cc70da3eabc6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.416959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9411f-8af0-4fba-b91f-cc70da3eabc6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.417365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9411f-8af0-4fba-b91f-cc70da3eabc6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.422985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 308/631] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:46.424309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.424769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.425186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.445093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:46.446547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:46.455703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c51ce7-e8aa-4256-a930-1d7c49d485d0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.494094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.496127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.496690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.497222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.511501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:46.513336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:46.518959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:46.519952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:46.520872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:46.525205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 324/632] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:46.525205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.540670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 309/633] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:46.542583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.586247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.638194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf938ee-5cdb-4306-8728-ec8aeeecbbc0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.671106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f778293-4b14-4e35-b01b-9b202f42e610 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.675102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.675724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.675999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.676265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] PATH_INFO: `/v3/users/glance` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.684988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.686638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:46.702337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:46.703310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 16:25:46.704261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:46.708037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 325/634] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:46.709000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.717476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.718335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.718785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.719260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.720210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:46.720710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.722414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.732535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 310/635] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.733743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.792226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.826138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33803-a280-479b-9a28-984aba0620ee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.843428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:46.857680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aaa99-3972-4e6e-98a1-e03e88ba1bc0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:46.873243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.874368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.874789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.875443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.899966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:46.903301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:46.903977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:46.904602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:46.904875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:46.905604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:46.905903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:46.906191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:46.907073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:46.907941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:46.908622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:46.908693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 326/636] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:46.908693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:46.935927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 311/637] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:46.935927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.026722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f68b-bca1-43f9-adb1-864d08358708 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.042902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2eb2d3-d122-473b-801c-7b8b98b15b43 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.055696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.063456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.064087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.064441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.064766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.079169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.080682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.082371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385f8c-c041-42db-8046-d411b801387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.083003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385f8c-c041-42db-8046-d411b801387a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.083284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385f8c-c041-42db-8046-d411b801387a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.083555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43385f8c-c041-42db-8046-d411b801387a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.095721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43385f8c-c041-42db-8046-d411b801387a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-43385f8c-c041-42db-8046-d411b801387a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:47.100529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:47.101488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:47.102947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:47.103615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:47.107897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:47.109001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:47.109001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.109001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:47.109169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 312/638] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:47.109169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385f8c-c041-42db-8046-d411b801387a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.109303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 327/639] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:47.111210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.139571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.213399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43385f8c-c041-42db-8046-d411b801387a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.218065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a21ec-cbca-41a0-8fd0-6ad5b1e0f61e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.250232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.250841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.251117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.251543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.252101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:47.252368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.253763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.256239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.256859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.257134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.257397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.262151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 313/640] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:47.262749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.295252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.333763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.335228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.360600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.375425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609555c7-1b7d-462b-8cd0-9f3e8584d09f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.410826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.411433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.411710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.411980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.412678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:47.412984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.414368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.423445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 314/641] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:47.424036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.445108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 328/642] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:47.446406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.526677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.536665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f292c4f-78ed-4392-93f4-6434263a2acc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.555626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc35d-d0ca-4557-9d6d-7eb3d315c935 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.575365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.576062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.576444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.576779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.577845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:47.578226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.579518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.581179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.596090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 315/643] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:47.596090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.597144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.597848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.598202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.598543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.610663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.612453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:47.628804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:47.630041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:47.630947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:47.630947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.630947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:47.630947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 329/644] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:47.631190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.706204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01c574-2764-46d9-a9e4-9091c2f7dd68 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.735739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b1783-e7ab-46e0-b031-e95dac97d86a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.740383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.741124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.741503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.741877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.753247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.754846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:47.770136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:47.771254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:47.772621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:47.772621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.772621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:47.777854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.778003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 316/645] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:47.778092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.778175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.778255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.778255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.782277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.789553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.791024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:47.809145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:47.810351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:47.816210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:47.816210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:47.816210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:47.816210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 330/646] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:47.816210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.846758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:47.891526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d542d20-54bf-49b6-a8e1-2225741c41aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.928044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2185d-4593-4bd9-a674-9ab9628e7008 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:47.931346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.932202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.932722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.933032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.940792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.942264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:47.954030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:47.955080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 317/647] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:47.956032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:47.969162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:47.969771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:47.970053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:47.970320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:47.970960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:47.971260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:47.972640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:47.983642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 331/648] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:47.984464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.019319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.057780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea1bca-69b2-4bc9-a9f0-65d8e464da4a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.071966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.084095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001452-6b4a-471d-9357-d2abfc7c2e08 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.092426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.093037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.093315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.093588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.094234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:48.094536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.096021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf328b-8568-409b-a9f7-d389765d2c7f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.108157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 318/649] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.109417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50384667-c08c-46ec-b7a5-eb8532178f18 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.109704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50384667-c08c-46ec-b7a5-eb8532178f18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.109973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50384667-c08c-46ec-b7a5-eb8532178f18 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.111169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 319/650] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:48.111865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50384667-c08c-46ec-b7a5-eb8532178f18 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.123979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.124609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.124885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.125152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.125790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:48.126086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.127437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.139135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 332/651] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.139917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.214707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50384667-c08c-46ec-b7a5-eb8532178f18 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.241603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.241980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f73373-2de4-40a3-86ff-419245699641 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.252609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.253215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.255119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.255119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.255119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:48.255119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.256161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.273322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 320/652] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.274236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.278622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.278970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.279518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.279790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.287755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.289216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b679877c-06de-4936-b68d-c69626b5996f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:48.300987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:48.302093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:25:48.303487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:48.304488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 333/653] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:48.305901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.306252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.307002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.325540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:48.326660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:48.328667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.377166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c69060-586a-4379-81e7-2f66b92482c2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.396301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 334/654] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:48.397154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.423789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.424520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.424870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.425199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.432581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.434103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:48.446211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:48.447353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 321/655] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:48.448535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.494754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.514480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0a3ff-3804-4e93-b842-1c523dbeaffb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.549362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.549982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.550260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.550530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.553707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2797-67f2-4741-bda4-93244a721bd6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.561077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.562623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.564580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:48.574686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:48.575653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:48.576619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:48.580565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 335/656] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:48.581318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.592662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.593273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.593539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.593810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.663173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.664743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.689364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49018476-72e9-4f4b-a08e-634796484afe None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.707207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.723245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.723849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.724131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.724432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.725241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:48.725543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.726976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.745170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 336/657] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.745983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.771395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 322/658] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:48.772262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.802261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.845016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc110ff-3fc7-45e8-b110-fb51cbe8333d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.871523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f2a6b-e41a-4975-9db5-d30ee38cf113 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:48.878618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.879215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.879494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.879761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.880513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:48.880814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.882236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.900294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 337/659] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.901733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.911748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:48.912896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:48.913466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:48.913757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:48.914449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:48.914749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:48.916222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:48.928627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 323/660] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:48.929400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:48.932542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:48.999421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c59694-ad57-4770-b52a-53fd540b142f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.014023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.027618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6e58a-e45e-47dc-9c6b-bd68bc921b8c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.033858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.034489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.034766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.035034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.047440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:49.052785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:49.054523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 338/661] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:49.056052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.067911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.068557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.068848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.069106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.139530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.141100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.142978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.161659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcd0f5-6c3d-4c53-b537-f1476c0e209b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.195559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.196182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.196492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.196767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.207848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.209318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:49.224403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:49.225525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:49.226633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:49.226633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.226633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:49.226633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 339/662] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:49.226633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.243820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 324/663] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:49.245388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.255226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.327683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31536ebf-ad8d-45cf-81cb-43e22738d6b3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.345160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465980-e069-4a78-9466-673a73a131f2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.346654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.363322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.363922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.364201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.364510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.380054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.380685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.380961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.381227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.381944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=43cdc8de558d4087bed8a3bb48cf4bfd) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:49.382243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.383676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.389651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 325/664] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/role_assignments?user.id=43cdc8de558d4087bed8a3bb48cf4bfd&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:49.390361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.443934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.445607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.474679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.514485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61236c1-3531-401d-8b7c-f989f647faaa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.556688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.557765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.558094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.558315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.559010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:49.559310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.560150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 340/665] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:49.560757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.561033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.573322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 326/666] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:49.573905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.606840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.684141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603b8a5-e284-4bdf-89e0-f7665aa8bfb5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.684705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ad8a2-c96b-4f6c-a237-57da8e951087 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.712714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.719453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.720072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.720363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.720636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.724569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.724569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.724569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.724569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.724870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:49.724870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.726155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.731202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.732692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.742766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 341/667] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:49.742766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:49.752049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:49.753182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:49.754078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:49.754078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.754078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:49.754078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 327/668] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:49.754078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.837155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:49.850684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459349a5-87a0-481d-8b30-5cd7e6ae0cd4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.857347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a222-0048-41ba-89f9-179e7f8ed403 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:49.892052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.892706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.892980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.893245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.902833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:49.903522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:49.903798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:49.904087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:49.909144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.910606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.916155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:49.917642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:49.927460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:49.928508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:49.929399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:49.929399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.929399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:49.929399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 342/669] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:49.930068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:49.934556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:49.935463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:49.936367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:49.936367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:49.936367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:49.940273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 328/670] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:49.941128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:49.977136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.042107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575b8cd-f2bd-4aac-9937-74005a35db59 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.052708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86d263-33fa-4cdd-a160-bcde1bcb7791 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.078755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.079725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.080056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.080329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.085662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.087533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.089624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.091401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.092370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.092810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.093236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.096332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.096332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.096332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:50.100808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:50.101678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:50.102764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 343/671] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:50.103306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.115760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 329/672] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.116778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.210654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.215879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe1d13-b08b-4929-ba59-4371b42ff4a5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.230847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b57f28-4285-417e-b8d3-d97315047d29 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.252779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.253392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.253797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.254325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.255463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.256009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.258604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.277672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 344/673] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.277672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.281053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.282263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.282263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.282263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.282917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.283214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.284629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.300665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 330/674] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.301659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.367468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.395693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686c96-0c62-4fbb-ac1e-3d2356313b3a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.417106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74947a9-84ae-4e00-a810-9e074594bd77 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.433613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.434446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.434694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.435038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.435863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.436173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.437635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.450439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 345/675] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.454603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.455312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.455689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.455962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.456071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.463927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.465512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.474833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:50.478008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:50.479713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 331/676] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:50.481127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.562891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb16ca-2198-4449-abed-73cd2808e091 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.587448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8175c9-ddfe-4465-bbcc-6282bb59a693 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.598712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.599459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.599741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.600012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.607374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.608928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.618924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:50.622807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:50.628831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 346/677] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:50.629727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.630314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.630314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.630661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.630780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.641061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.642542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:50.654496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:50.661195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 332/678] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:50.662121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.721534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.745728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1237ff8-9589-4941-9943-7e8dd3d8ecc6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.765913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c870863-a36a-4d6d-bf7a-ce86ea2ea1db None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.782661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.783258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.783533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.783800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.784539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=4542cf167cc24738b441af6530dead3c) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.784842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.786249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.800409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 347/679] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/role_assignments?user.id=4542cf167cc24738b441af6530dead3c&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.800409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.814023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.814982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.815773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.816115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.816953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.817452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.819742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.840507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 333/680] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.841910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.881784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:50.932443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ade255-8437-4b70-8ee9-c102d91c02ce None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.976252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:50.977240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:50.977518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:50.977794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:50.979044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:50.979344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:50.980819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:50.991798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08b25-2bf7-4d55-860e-090a6cf18fbe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:50.993561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 348/681] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:50.995073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:50.997831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.039196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.039813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.040093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.040405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.042395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.042716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.044185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.065346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 334/682] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.066256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.120220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00fd20-1069-41f5-84ce-5d3139db998e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.163515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.164676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.164961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.165234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.165991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=ef06aba40382479289b1e9e19395a2be) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.166834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.168737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.176964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 349/683] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:50 2022] GET /identity/v3/role_assignments?user.id=ef06aba40382479289b1e9e19395a2be&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.177797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.184421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.195396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32df127-6487-4c92-be46-7007095f27f7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.237908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.238451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.238779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.239060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.239808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=9bde9ceb3c7c46298626580b62e0ec01) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:51.240122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.241660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5504ae-f15b-4150-b4ef-d9782603c045 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.249584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 335/684] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/role_assignments?user.id=9bde9ceb3c7c46298626580b62e0ec01&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:51.310417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.351675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66631fdb-a3cd-45f1-9692-c3dd998c4291 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.393578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.394201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.394477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.394746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.430753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048186d-951c-4f05-85d9-b9eb66cee841 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.431181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048186d-951c-4f05-85d9-b9eb66cee841 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.432284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048186d-951c-4f05-85d9-b9eb66cee841 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.434131 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 336/685] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:51.439859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.440263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.440672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.470048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:51.471406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:51.476786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.478353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.559466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.570673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 337/686] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:51.602767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 350/687] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:51.609152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.714670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:51.736745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d190207-40ad-45f8-8ddb-2e01c32f93c9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:51.798325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:51.798962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:51.799289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:51.799562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:51.821540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:51.823034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:51.848355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:51.850686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:51.852387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:51.852387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:51.852387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:51.852387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 338/688] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:51.855856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:51.925048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.031008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff50c5c-6a99-462d-bcd4-a4de621cc3bf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.096991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.097517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.097791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.098061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.098715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:52.099015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.100405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.123324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 351/689] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:52.129950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee928a2e-46ea-49f7-9d54-c69ae8781e12 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.134102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.181045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.182531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.182807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.183077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.190558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.204744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.210189 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.211675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:52.239955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:52.247466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:52.248380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:52.248380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:52.248380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:52.248380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 339/690] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:52.248380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.334784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ce554-dabe-4bd4-a8f5-95d7068ac1f0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.407978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.408561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.408838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.409106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.423924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.425411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:52.451628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:52.452586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:52.453482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:52.458489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 352/691] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:52.459126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d924c0-a704-4b5d-93c2-baa6ff70797e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.461527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.537172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.537789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.538068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.538325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.538977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:52.539274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.541229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.571034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 340/692] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.576551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.640751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.671009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caec12-aea4-45b8-b1d9-d8c3fd41e31c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.676912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:52.746710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.747329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.747606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.747878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.748602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:52.748907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.750347 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.771627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97ec09-c339-45a1-9dfe-c1629e7baba7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.789915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 353/693] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:52.797435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:52.812577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22260a1a-55b1-4d7d-b723-cdd9c3cab246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.813191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260a1a-55b1-4d7d-b723-cdd9c3cab246 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.813466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260a1a-55b1-4d7d-b723-cdd9c3cab246 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.813867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260a1a-55b1-4d7d-b723-cdd9c3cab246 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.814511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22260a1a-55b1-4d7d-b723-cdd9c3cab246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.815976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22260a1a-55b1-4d7d-b723-cdd9c3cab246 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.905449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 341/694] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:25:52 2022] POST /identity/v3/users => generated 254 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:52.939827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ead0-ec15-4719-97fb-61b8eea0cd7f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:52.977856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:52.978543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:52.978825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:52.979097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:52.979822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=1f762db26df84f5faf7e6778383e2870) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:52.980123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:52.981587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46331ed1-7f96-46b2-9fac-74a37e489c9a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:52.988531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 354/695] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:52 2022] GET /identity/v3/role_assignments?user.id=1f762db26df84f5faf7e6778383e2870&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:53.181936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52c82-86f1-4aaa-b974-5730144ed56b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.182768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52c82-86f1-4aaa-b974-5730144ed56b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.182768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d52c82-86f1-4aaa-b974-5730144ed56b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.184257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 342/696] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:53.189564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46e970-11ab-475d-9278-017783b0152e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.189900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46e970-11ab-475d-9278-017783b0152e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.190097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46e970-11ab-475d-9278-017783b0152e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.214009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-de46e970-11ab-475d-9278-017783b0152e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:53.215232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-de46e970-11ab-475d-9278-017783b0152e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:53.218282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:53.293325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 355/697] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:53.536282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd601-bf45-4435-97b1-2997e7f93978 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.536700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd601-bf45-4435-97b1-2997e7f93978 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.536870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd601-bf45-4435-97b1-2997e7f93978 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.538416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 343/698] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:53.544662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.544954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.545227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.567308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:53.568839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:53.575373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd601-bf45-4435-97b1-2997e7f93978 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:53.656649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 356/699] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:53.704919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd601-bf45-4435-97b1-2997e7f93978 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:53.707600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:53.741421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abd49c-ddb0-4f11-b1e2-293e8350853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:53.742300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd49c-ddb0-4f11-b1e2-293e8350853d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.743477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd49c-ddb0-4f11-b1e2-293e8350853d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.743477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abd49c-ddb0-4f11-b1e2-293e8350853d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.757340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abd49c-ddb0-4f11-b1e2-293e8350853d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d3abd49c-ddb0-4f11-b1e2-293e8350853d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:53.764839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:53.766008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:53.767047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:53.769507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 344/700] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:53.776046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:53.877547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5692e8-59ee-4d21-a88e-df784f6d4f84 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.878060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5692e8-59ee-4d21-a88e-df784f6d4f84 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.878479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5692e8-59ee-4d21-a88e-df784f6d4f84 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.880006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 345/701] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:53.881255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:53.886255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.886763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.887191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.917448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:53.918652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:53.944296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac2670-8aa6-4d6a-b725-adc6f64e058e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:53.985268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:53.985877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:53.986161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:53.986436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:53.987111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:53.987416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:53.988850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:53.998527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 357/702] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:54.003796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.062135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 346/703] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 16:25:54.070132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.230957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b3b92-f7fc-44a5-8fd4-75f1e60f2875 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.277139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.284966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00017bf9-d58d-4803-91dd-e8af8b7ce05f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.296530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.313341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.313989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.314269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.314539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.327934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.329534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:54.344540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:54.345530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:54.346436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:54.346436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:54.346436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:54.346436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 358/704] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:54.351411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.391063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.391747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.392019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.392332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.392957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.394336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:54.412719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:54.413778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.414688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 347/705] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:54.415519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.485364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b22844-3c18-4c8c-a4d0-96c94e113cce None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.535003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.535741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.536602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.536872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.537569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:54.537914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.539692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.555853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662ea77-6344-40d0-b8ff-a8dd8912e7c1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.558754 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 359/706] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:54.559826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.612629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.614002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.614486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.615012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.616143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.618752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.624318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:54.634823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:54.642280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.643152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 348/707] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:54.644011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.670946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.695627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa51aa3-02ba-4d0e-a4a3-41677eb8efba None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.733883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.734591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.734925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.735267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.735998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:54.736322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.737924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.743264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 360/708] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:54.744907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.755474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec3daf-c351-4de1-8111-c5e41c1078fd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.794763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.795713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.796048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.796426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.806791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.809177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:54.824622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:54.825595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:25:54.826532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:54.829892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 349/709] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:54.830867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.864603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4e9bf-986a-4fff-b181-84b58e5fa21d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:54.878201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.915090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:54.916919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:54.917665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:54.919058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:54.919778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] RBAC: Adding query filter params (name=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:54.920288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:54.922081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:54.932464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 361/710] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:54.933851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:54.964978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:54.990149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac6233c-7855-4bf2-9d0e-ab82e0e83a9c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.038754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.039677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.041403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.041855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.043373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:55.043889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.046259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.054201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 350/711] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:55.055234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.082759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f9d47-f1e1-49ee-a50f-195540162596 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.170722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.171551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.171866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.172183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.173042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:55.173440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.175114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.193272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1c86-0238-4fcf-936a-d71ce2f7fe9e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.212686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 362/712] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:25:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:55.219263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.243208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.245138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.245722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.246332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.247972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:55.250190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.251665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26931b1-4157-4c26-9a20-0d6d15e6d810 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.258566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 351/713] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:55.275975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.295818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a6148-3f74-4a48-bef9-fd6e3a968930 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.296390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a6148-3f74-4a48-bef9-fd6e3a968930 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.296390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a6148-3f74-4a48-bef9-fd6e3a968930 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.298036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 352/714] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:55.305110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637b0e0-f09e-49c0-9c81-5910e8d77e7e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.305430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637b0e0-f09e-49c0-9c81-5910e8d77e7e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.305658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637b0e0-f09e-49c0-9c81-5910e8d77e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.341116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6637b0e0-f09e-49c0-9c81-5910e8d77e7e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:55.342345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6637b0e0-f09e-49c0-9c81-5910e8d77e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:55.346555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.380952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03412752-59ad-4f00-b413-4ede81068df9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.423201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.424289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.424819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.425292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.481581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 353/715] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:55.518949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.520327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.554056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.606129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8729b9-82c4-445f-82f2-0f3e0589329d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.606422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8729b9-82c4-445f-82f2-0f3e0589329d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.607199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8729b9-82c4-445f-82f2-0f3e0589329d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.612389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 354/716] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:25:55.614255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.614534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.614803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.637059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 363/717] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:55.643269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.661403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:55.662613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:55.780975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb57fc-6bb9-4de4-833f-d7d85593d458 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:55.811411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:55.818001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:55.818634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:55.818930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:55.819182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:55.826256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 355/718] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:55.827732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.831743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:55.832999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:55.857040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:55.858101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:55.858981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:55.858981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:55.858981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:55.861229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 364/719] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:55.868263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:55.960744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.005308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ff94-9a1f-40c7-be56-bf73c94de186 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.030438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6ecab-3399-4fe2-99f6-675f3248f375 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.041821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.042348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.042627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.042903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.043523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:56.043822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.045201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.058287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 365/720] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:25:56.064399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.107171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.107807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.108083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.108429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.125636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:56.130317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:56.139836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 356/721] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:25:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:56.140704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.141451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.222537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170a07a-4a94-459b-9df6-ade7c6cfb5ec None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.250345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fe9e4-eef4-4dd6-96fc-0bb8c307622a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.270504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.271143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.271427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.272221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.281167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.283923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.285949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.287312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.288121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.288503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.288857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.300986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:56.302063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:56.303155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:56.304243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:56.306863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 366/722] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:56.307908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:56.308429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:56.309504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 357/723] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:56.315283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.438270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe95ce-7b4e-4eb8-b456-6607a5c7b30e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.450311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f432f6-12a1-462b-8cae-00430c2d3350 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.453389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.485899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.486870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.487375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.487803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.488828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:56.489350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.490142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.490741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.491017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.491286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.491657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.491936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:56.492235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.494349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.506981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 358/725] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:56.507405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 367/725] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:56.508374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.508491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.568725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.628057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f45b-4690-4492-8828-16e403c74367 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.675430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.676400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.676826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.677275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.678501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:56.679060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.681021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.698212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 359/726] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:56.698212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.762205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36625e-5e4e-4f79-8f7b-617f892d9ce4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.810703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.811440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.811788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.812124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.825860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05106f1-c659-4303-be15-fea70639cab5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:56.826605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.829317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.839710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:56.853784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:56.861520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:56.862458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:56.862458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:56.862458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:56.862458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 368/727] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:56.862458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:56.874741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:56.875361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:56.875638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:56.875912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:56.876927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=4542cf167cc24738b441af6530dead3c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:56.877237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:56.879486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:56.880634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:56.889159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 360/728] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/role_assignments?user.id=4542cf167cc24738b441af6530dead3c&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:56.890580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.020399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5648-8238-45af-9f0c-52646792add4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.062657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.063278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.063551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.063821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.064514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.064812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.066218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.082231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 361/729] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.088209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.107643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea314ded-7195-4490-a251-27e3f13fab6e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.133317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.151506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.152921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.153400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.155282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.167823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.171945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:57.194241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:57.201666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:57.203575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:57.203575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:57.203575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:57.203575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 369/730] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:57.209408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.217100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f5c81-4194-4003-96a7-e3cf8b6098eb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.256588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.257648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.257930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.258200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.267354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.269006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:57.284227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:57.286654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:57.289056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 362/731] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:57.293606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.339238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.382702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29edb3-56b1-4eb6-b313-2261d047db85 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.413565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.423201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98a66d-da9a-40e5-97eb-f6346ce0cdec None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.428829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.429520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.429869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.430207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.431012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.431969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.433910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16deedd-4eb0-4693-b929-1164dea6f611 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.450707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 370/732] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.451778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a81168-1507-463d-919e-db72731dad18 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.452062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a81168-1507-463d-919e-db72731dad18 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.452360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a81168-1507-463d-919e-db72731dad18 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.457239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 371/733] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:57.457239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a81168-1507-463d-919e-db72731dad18 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.480391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.481497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.482060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.482536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.483556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.484088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.485997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f67b5-f4c4-4a1d-8726-96206c6c5352 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.507048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 363/734] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.508848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.509348 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.509935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.530483 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:57.532068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:57.578482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a81168-1507-463d-919e-db72731dad18 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.605212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 364/735] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:57.605771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.613392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.625128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.626064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.626513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.627001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.638557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.640398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4ea63034-88a4-48e2-a0a4-8341d8d7f9d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:25:57.657756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:57.658925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:25:57.660092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:57.664677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 372/736] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:57.667005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da74be2-7a24-4a15-9a6d-02bd3897de5e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.668668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da74be2-7a24-4a15-9a6d-02bd3897de5e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.669161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da74be2-7a24-4a15-9a6d-02bd3897de5e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.671799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 373/737] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:57.672871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da74be2-7a24-4a15-9a6d-02bd3897de5e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.708387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.725248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa7b47-b3b2-4dae-82f2-3d82f78e454f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.769047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.769646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.769921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.770187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.803040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da74be2-7a24-4a15-9a6d-02bd3897de5e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:57.839570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:57.840218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.840535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.840828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.841528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:57.841827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.843304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89507705-a514-472c-9bf6-894e413c9209 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.866028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 374/738] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:57.866028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:57.866028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:57.866028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:57.871089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:57.873672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:57.882782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:57.887036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:57.933467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.958592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 375/739] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 16:25:57.959729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:57.972566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:57.994105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 365/740] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:57.995040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.098620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d7203-0ed6-43f9-99e6-5655931487bb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.116139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b20af-d17c-477e-bfde-04fdef5b0cea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.151784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.152938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.153374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.153773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.154705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.157057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.160787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.161514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.161883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.162248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.219308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.234850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.236302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.270854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 376/741] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:25:57 2022] POST /identity/v3/services => generated 251 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:58.273615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.329917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.359350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 366/742] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:25:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 16:25:58.360976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.449647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6551b-5f4d-412f-92bf-26ed0409fcb2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.478301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.497173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a603f90-fca6-4753-8f5e-fd6a2530dcf9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.506939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.507594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.508408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.508694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.523394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.525695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.535691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.536738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.537226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.537863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.538701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.540843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:58.543665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:58.544832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:58.545722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:58.545722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:58.545722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:58.545722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 377/743] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:58.545722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.624881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 367/744] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:25:58 2022] POST /identity/v3/services => generated 240 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:25:58.626139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.669645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.702176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667f55-3703-402f-8978-563bca18b905 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.792160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.792830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.793104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.793370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.808961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.810820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:25:58.826412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:58.827442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:25:58.828441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:25:58.828441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:25:58.828441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:58.828441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 378/745] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:58.833709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d5942-9f1d-495e-87f2-a1755efde631 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.837394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.877275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.877893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.878170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.878434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.879087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:58.879537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.881175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:58.904204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 368/746] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:58.910036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:58.914280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.953090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c56d-1d47-48f3-ad07-6428a8468430 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:58.983455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:58.988400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:58.989018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:58.989296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:58.989569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:58.990220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:58.990518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:58.991894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.010809 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 379/747] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.012099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.029886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b018d6-ce95-4603-8250-7432c3f1c854 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.072759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.073412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.074077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.074355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.083725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.085518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-46c7f8ad-dc3b-4d84-b11b-bd2a428b617f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:59.105306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:59.106230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:59.107124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:25:59.109665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 369/748] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:59.111594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6536b4-bc26-44fb-a990-b8f47bec1fda None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.112016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6536b4-bc26-44fb-a990-b8f47bec1fda None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.112446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6536b4-bc26-44fb-a990-b8f47bec1fda None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.114133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 370/749] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:25:59.115740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6536b4-bc26-44fb-a990-b8f47bec1fda None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.146330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d0a6-8c72-4734-bb72-66a926dd0bac None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.194427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.195055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.195337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.195593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.202394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.205727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.207286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c8311e2b-e18a-46b9-ad33-90ee3f0e7d4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:25:59.222185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:59.223163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:25:59.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:59.228437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 380/750] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:25:59.230228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.230512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.230783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.250152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:25:59.251444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:25:59.258650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6536b4-bc26-44fb-a990-b8f47bec1fda None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.303594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.304187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.305666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.306405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.306913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.308089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.308887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.310562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.326320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 381/751] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:25:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:25:59.327281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.327863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 371/752] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.340053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.457036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7336bd-a87a-44c8-989d-753ef03d087d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.484306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.511008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.511897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.512168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.512926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=270fe102379a4346ae009a4545865c28) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.513233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.516492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d752e1-6706-48de-9857-2afe84e9a388 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.519146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.538561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 372/753] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/role_assignments?user.id=270fe102379a4346ae009a4545865c28&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.590711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.591345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.591629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.591901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.592636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.592942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.594386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.618110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 382/754] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.624443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.635850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.733075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:25:59.754238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7310d-086f-44dd-97bc-1434adf3200f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.796850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.797616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.797898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.798177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.799522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=1f762db26df84f5faf7e6778383e2870) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:25:59.799910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:25:59.801869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:25:59.812435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 373/755] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/role_assignments?user.id=1f762db26df84f5faf7e6778383e2870&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:25:59.826674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7290e-1368-4acd-9adc-4cf48b8296b6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:25:59.866522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:25:59.867210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:25:59.867496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:25:59.867766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:25:59.879236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:25:59.886246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:25:59.887276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:25:59.888117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:25:59.889311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 383/756] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:25:59.895748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:25:59.953771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.037076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc03f94-7dd3-4c0b-95c5-0b93d6f97bf6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.083862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.084520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.084798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.085064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.085740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:00.086040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.088112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.100048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 374/757] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.106201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.224315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d1474-fc9a-4861-b96c-d5872ba76a33 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.225913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.261993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.262639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.262919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.263190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.274246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.276251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cc3c699b-a9d8-4be0-9a79-18a87e5e41b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:00.296794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:00.297770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:00.298688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:00.298688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:00.298688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:00.298688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 384/758] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:00.303723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.429431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e40b7-1a26-419d-8562-c44734c3dc7f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.442763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ed03f-3b5e-4b36-b46e-a9cf2f00c58b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.443038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ed03f-3b5e-4b36-b46e-a9cf2f00c58b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.443311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ed03f-3b5e-4b36-b46e-a9cf2f00c58b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.444697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 385/759] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:00.449105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.451073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.451573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.452069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.472025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.472616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.472897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.473162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.474159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:00.474159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.475502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.488647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 375/760] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.492472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:00.494461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:00.495397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.632771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 386/761] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:00.654684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438089a5-091a-42bf-a415-27343cfdfe5a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.698957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.699583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.699856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.700118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.707888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.709366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:00.720580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:00.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:00.722477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 376/762] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:00.729817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.840376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:00.865528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166ee8b-8cc8-40d9-a4d6-2080306ea8ac None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:00.909700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:00.910309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:00.910581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:00.910844 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:00.911630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:00.911933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:00.913417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:00.937779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 387/763] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:00.943204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:00.989986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.074685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b1a90-24f9-458c-8361-4403439e9c1f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.093843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.122213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.122851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.123129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.123393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.136711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.138587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:01.158689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:01.160232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:01.161816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:01.161816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:01.161816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:01.161816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 377/764] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:01.172319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.238469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de8488-7d64-41b6-9c99-90dedbe70a7b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.279357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.279988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.280266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.280579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.281157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.282519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4318214c-b818-4fb6-98e4-4fb4d768124e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:01.293914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:01.296248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:01.298286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 388/765] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:01.300249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1564b-35f5-45c1-8246-9d874c0a18e3 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.300249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1564b-35f5-45c1-8246-9d874c0a18e3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.300249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1564b-35f5-45c1-8246-9d874c0a18e3 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.308817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 389/766] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:01.308817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1564b-35f5-45c1-8246-9d874c0a18e3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.313169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.324071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.343455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb79f-ea6f-4698-94e0-0dbcb7f392ad None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.378046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.378700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.378980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.379343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.380082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.380422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.381871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238fb09-1a65-4425-a154-0ad68b6e5764 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.395659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 378/767] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.397408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.397792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.398164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.419292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:01.420649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:01.426869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1564b-35f5-45c1-8246-9d874c0a18e3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.468063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.468700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.468978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.469243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.469884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.470184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.471570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.477008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 390/768] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.478581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.512509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 379/769] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:01.513736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.551126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.583298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f43de5-9131-47c4-9d20-25182f94118b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.626308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.626665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.626913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.627184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.640812 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.642257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.654129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:01.658820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:01.659736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:01.660676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:01.666946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 391/770] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:01.669691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.685287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d883b2-0578-446d-8f75-8ea40eaa8de7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.734126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.735290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.735819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.736300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.737164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.738317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.739915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.758461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 380/771] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.764383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.795476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343573ff-00f5-48a4-a671-8519902f6069 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.828843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:01.843271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.844214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.845181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.845464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.846402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.846906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.848755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.873071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 392/772] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:01.878497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.903178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598f1f1-7137-4d10-a6b2-263d75a06fa4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:01.942709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:01.943240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:01.943514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:01.943780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:01.944515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] RBAC: Adding query filter params (interface=public, service_id=0e3dc36946cf4b99992f87002cec95f6, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:01.944951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:01.946611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:01.954901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 381/773] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/endpoints?service_id=0e3dc36946cf4b99992f87002cec95f6&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:01.955841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:01.959594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.007455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de9b0d-11a7-474c-a553-0da3f611c8e5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.047828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.048950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.049438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.049964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/4542cf167cc24738b441af6530dead3c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.052300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=4542cf167cc24738b441af6530dead3c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.054085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.065811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036795b-f584-4ecd-8f17-44e054f52515 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.103162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.103802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.104079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.104370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.104955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.106350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:02.115214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:02.116265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:02.123201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 382/774] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:01 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:02.124004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.162274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 393/775] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:01 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/4542cf167cc24738b441af6530dead3c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:02.163719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.200269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.226449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.241659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bed1cc-7a9d-41b1-91a7-d82cfb898470 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.280970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018004e4-b150-4eb3-9c22-e723cfe2d2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.281591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018004e4-b150-4eb3-9c22-e723cfe2d2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.281863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018004e4-b150-4eb3-9c22-e723cfe2d2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.282130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018004e4-b150-4eb3-9c22-e723cfe2d2a3 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.282855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018004e4-b150-4eb3-9c22-e723cfe2d2a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.284300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018004e4-b150-4eb3-9c22-e723cfe2d2a3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.286968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd5c5f-e1f2-4166-bacb-b5f27f2346e5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.290094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 383/776] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:02.291595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50ad3-4f85-46e7-8848-976c09025deb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.291948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50ad3-4f85-46e7-8848-976c09025deb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.292295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd50ad3-4f85-46e7-8848-976c09025deb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.297984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 384/777] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:02.297984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b3f76-a780-403f-ae1f-64c984a3f774 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.297984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b3f76-a780-403f-ae1f-64c984a3f774 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.297984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b3f76-a780-403f-ae1f-64c984a3f774 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.301850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 385/778] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:02.303164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.303443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.303713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.324032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:02.325291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:02.353084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.353534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.353669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.353934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.354583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] RBAC: Adding query filter params (name=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:02.354883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.356295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323576ee-7d0f-4778-95d3-68456c67b46b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.362576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 394/779] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:02.363971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.364256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.364561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.388539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:02.389925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:02.434070 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 386/780] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:02.440370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.485952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 395/781] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:02.523872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.569585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07988b-a53c-40d4-bbe7-c1bf0eb022ee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.722788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.723693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.724108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.724539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.725334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:02.725784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.727301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.733464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 387/782] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:02.740427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.751101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.843006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.867789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7406c-524e-4054-b0f4-f99ee0243679 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.904708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:02.907306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ea900-1b5c-4407-8159-5cc048d477fe None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:02.915941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.916916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.917740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.918278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.931917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:02.938596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:02.939535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:02.940397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:02.942018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 388/783] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:02.944392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:02.958833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:02.959959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:02.960313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:02.960693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:02.961476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] RBAC: Adding query filter params (interface=public, service_id=70f130bb413e4d5daeef76813717b475, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:02.962786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:02.964613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:02.974826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 396/784] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/endpoints?service_id=70f130bb413e4d5daeef76813717b475&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:02.976386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.071006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f814e1-1316-47c6-8661-7546d66eaa94 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.096547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d055f85-1bc2-40bc-86a1-7367ce497010 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.107298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.121515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.122600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.123057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.123811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.135286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.135969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.136314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.136682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.137565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:03.137874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.138417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:03.139464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:03.143993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:03.144984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:03.145807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:03.152827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 389/785] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:03.152827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.158184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 397/786] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:03.158184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.214832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.287706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def00fc-5e83-49fe-9296-164e9e3176c0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.292989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8a22d-10fb-4a18-9279-cce99a834506 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.327565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.328284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.328689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.329066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.329835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:03.330242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.331716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.341016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.341801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.342213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.342639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.343374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.344815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 398/787] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:03.345244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.345896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.352485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 390/788] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:03.353954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.416264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.480427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb41ee-17b8-4eec-b08e-469d9eea3740 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.495311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.527521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.529166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.529591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.530061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.546508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.548245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.558009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b893c-7be8-415e-8f2c-71476b1c7975 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:03.567591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:03.568669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:03.569554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:03.569554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:03.569554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:03.569554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 391/789] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:03.581760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.613744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.614382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.614657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.614924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.627145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.628705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:03.644547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:03.645573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:03.646532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:03.646532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:03.646532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:03.646532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 399/790] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:03.651560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.715992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85969f5c-03aa-480f-99d9-2f334c026ab7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.759254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.759863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.760576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.760855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.761509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:03.761962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.763598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.777457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 392/791] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:03.778516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.784609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:03.828243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa0efc-517e-47d1-98ff-399b83b34ae2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:03.856875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:03.957449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:03.958220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:03.958606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:03.959062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:03.959834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:03.960251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:03.961965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:03.988063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 400/792] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:03.990936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.028461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecdcec-0148-4912-a972-948eed8cc0dc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.068428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.069651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.070127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.070650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.088376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.088592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:04.113650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:04.114612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:04.115539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:04.117393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 393/793] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:04.120022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.218911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c1259-f550-4e3a-8452-bd17bae98777 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.240709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.262177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0559bf-3eeb-41f7-bdcd-6429dcaa93e6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.299868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.300523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.300799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.301194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.302067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:04.302371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.303812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.323774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 394/794] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:04.331396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.337406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.339951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.340233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.340541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.357698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.359950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:04.382329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:04.383472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:04.386191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 401/795] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:26:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:04.392798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.397433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.472655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66d0f9-a90f-43d8-9b05-b10866be545b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.504488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628ccce-0b2b-492a-a72c-e8b08d90628b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.509862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.510492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.510766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.511037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.522230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.526117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.526117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8eeccdb4-0b9e-4647-bca1-0a7ce8f99cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:04.542837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:04.545698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 395/796] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500d3e-26d8-4b2e-9f2f-7018de56bb84 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500d3e-26d8-4b2e-9f2f-7018de56bb84 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b500d3e-26d8-4b2e-9f2f-7018de56bb84 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.547853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 396/797] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:04.553841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500d3e-26d8-4b2e-9f2f-7018de56bb84 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.556591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.557327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.557609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.557986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.558739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:04.559045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.560540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6ff57-cfb8-4ce3-87f8-e1706a63c68b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.580849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 402/798] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:04.582316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4fde6-198b-40de-b6e8-994be89ad78a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.582610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4fde6-198b-40de-b6e8-994be89ad78a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.582938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4fde6-198b-40de-b6e8-994be89ad78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.601789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3ef4fde6-198b-40de-b6e8-994be89ad78a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:04.603349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3ef4fde6-198b-40de-b6e8-994be89ad78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:04.663681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b500d3e-26d8-4b2e-9f2f-7018de56bb84 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.666641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.684381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 403/799] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:04.684381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024563-5179-4c07-8256-7ca2f5e18837 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.684381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024563-5179-4c07-8256-7ca2f5e18837 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.684381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56024563-5179-4c07-8256-7ca2f5e18837 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.684381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 404/800] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:04.684381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56024563-5179-4c07-8256-7ca2f5e18837 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.714603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.715477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.716149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.716464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.717389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:04.717688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.719877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5009fe0-a8a5-4cde-b8c9-920a4b1df428 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.735328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 397/801] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:04.736799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.737081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.737353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.761869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:04.763314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:04.802436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56024563-5179-4c07-8256-7ca2f5e18837 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:04.833014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:04.860610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 398/802] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:04.861719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.865792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:04.868036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:04.868768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:04.869275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:04.885488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:04.887436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:04.910938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:04.912071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:04.913215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:04.913215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:04.913215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:04.914046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 405/803] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:04.920477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:04.970248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.039010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda54b9-6c2d-4e1f-a483-eaae50500d7f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.068964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe8485-034a-4431-9132-259b9113dd22 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.077124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.077735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.078010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.078274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.079019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:05.079317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.080787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.097693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 406/804] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.098539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.114294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.114917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.115186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.115448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.123483 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.124994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:05.137524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:05.138427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:05.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 399/805] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:05.140241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.189526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.234751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041889a6-78be-41aa-b3ed-df232c96a2d6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.239904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.252387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a5f4c-036e-4027-96dd-410bcb49bd66 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.281270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.282210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.282665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.283145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.292398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.293032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.293305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.293573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.295624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:05.300351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:05.300721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:05.301645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:05.302473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:05.303116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.307811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 407/806] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:05.307811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:05.314352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:05.315245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:05.316119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:05.320117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 400/807] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:05.321207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.416364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7088d2-263f-4d18-b252-907d434aace6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.435200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc416f6-6e79-440a-8664-64f004522cf3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.436860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.452556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.453087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.453361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.453626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.454313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:05.454614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.456136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.473158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 408/808] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.473158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.478003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.478241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.478487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.478747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.479324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.480719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:05.489798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:05.490862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:05.491669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 401/809] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:05.492512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.505861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.588923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85471f20-4104-4630-8e38-4dea8336c428 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.603009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b4a3a-bff3-4b0c-abf5-32a5639a5c33 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.624845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.625451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.625724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.625992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.626645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:05.626941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.628300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.636471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 409/810] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.637048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.642662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.643278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.643553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.643817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.644543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:05.644868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.646290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.659688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 402/811] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.660537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.687946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.723148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.738018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026bc69-4683-4eab-b443-63729268d4e2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.760304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004b892-47e4-49ab-b514-e097b176a889 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.771958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dafe5-e747-4092-a04d-8b2880808d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.772636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dafe5-e747-4092-a04d-8b2880808d04 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.772932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dafe5-e747-4092-a04d-8b2880808d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.773178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dafe5-e747-4092-a04d-8b2880808d04 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/1f762db26df84f5faf7e6778383e2870/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.775235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dafe5-e747-4092-a04d-8b2880808d04 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=1f762db26df84f5faf7e6778383e2870, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.776664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dafe5-e747-4092-a04d-8b2880808d04 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.794221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.794819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.795098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.795366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.795992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:05.796308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.797698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.802775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 403/812] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:05.802775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.850919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 410/813] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:05 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/1f762db26df84f5faf7e6778383e2870/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:05.852931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728addf9-4b6d-4740-8757-06b53a30fa81 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.853209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728addf9-4b6d-4740-8757-06b53a30fa81 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.853501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728addf9-4b6d-4740-8757-06b53a30fa81 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.856382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 411/814] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:05.856382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728addf9-4b6d-4740-8757-06b53a30fa81 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:05.911512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:05.922690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20feac-9580-4a32-97d8-0b138386dcd0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.964106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:05.964799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:05.965081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:05.965350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:05.977794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:05.979200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:05.996265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728addf9-4b6d-4740-8757-06b53a30fa81 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:05.997121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:05.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:05.998949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:05.998949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:05.998949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:05.998949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 404/815] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:05.999628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.038331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df319b3-37c6-4936-be5c-6ee780af26c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.038990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df319b3-37c6-4936-be5c-6ee780af26c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.039267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df319b3-37c6-4936-be5c-6ee780af26c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.039537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df319b3-37c6-4936-be5c-6ee780af26c4 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/270fe102379a4346ae009a4545865c28/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.055427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df319b3-37c6-4936-be5c-6ee780af26c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=270fe102379a4346ae009a4545865c28, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.057605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df319b3-37c6-4936-be5c-6ee780af26c4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.061239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.107533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f4081-bb7e-46b3-902d-689edc1783c6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.134721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 412/816] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:05 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/270fe102379a4346ae009a4545865c28/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:06.136087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.136401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.136858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.157272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.158380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.158849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.159235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:06.159416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.160271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:06.160614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:06.160974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.162679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.179030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 405/817] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Aug 24 16:26:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:06.179030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.193186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.276398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 413/818] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:06.276398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.331835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c884-2c98-4af7-a496-a317da2c5f45 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.371372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.372364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.372800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.373259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.374041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:06.374570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.376116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.387639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d9f2e-e2d3-45fc-b68d-e56117e18fb8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.390394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 406/819] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:06.405897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.424627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.429319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.429971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.430793 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.431236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.435951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.435951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.504387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.520602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 414/820] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:06.567144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b376-702f-455f-b8ec-1c6d5e2c048c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.621820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.627707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.629018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.629477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.629923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.641754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.643661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:06.662951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:06.663937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:06.665668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:06.670819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 407/821] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:06.677321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.794887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbf697-3ffb-4643-a81c-1dbd30a8a76d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.832504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.833060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.833336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.833440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fce66-31c0-4c2c-a460-444573baa95d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:06.833597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] PATH_INFO: `/v3/services/network` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.834177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.835596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:06.858349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:06.859412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:06.860248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 415/822] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:06.861369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:06.866480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.878120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:06.879016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:06.879477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:06.879974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:06.881833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:06.882350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:06.885730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:06.915669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 408/823] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:06.924363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:06.988091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.068524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb3464-a34c-439a-a79f-4a2c85d4d2ef None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.105408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.105940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.106222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.106493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.107130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] RBAC: Adding query filter params (name=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:07.107432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.108995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.119548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 416/824] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:07.121053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.142683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60327ddd-31f1-4de1-b3ab-9b79b090ac8a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.218170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.218785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.219055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.219319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.264295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfe1e-8f63-492c-9243-fd9b6d7182ab None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.284482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.312714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.332241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.332876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.333153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.333423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.334152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:07.334459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.335826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.350059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 417/825] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:07.350803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.352935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.357314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.514503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f5a1e-ef00-4a32-9c9f-3687fed7f8e0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.551905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 409/826] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:07.560478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.571573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.574099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.574383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.574651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.575319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.576709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.659298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.682878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 418/827] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:07.687911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53875aa-76e5-4786-b695-1a3ea6014646 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.739092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.739736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.740009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.740275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.751905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.757748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:07.779232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:07.780243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:07.781186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:07.781186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:07.781186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:07.781186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 410/828] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:07.786217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:07.831962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:07.945607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d6cd0-4730-4678-85a8-44e889d9672e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:07.989792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:07.990430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:07.990709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:07.990979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:07.991653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:07.991956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:07.994010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7609adb-789d-42fc-b44d-d5b337696950 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.007573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 419/829] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:08.017459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.096547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581692f8-83b9-4042-9cf2-b1ec4c9f9f17 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.097171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581692f8-83b9-4042-9cf2-b1ec4c9f9f17 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.097654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581692f8-83b9-4042-9cf2-b1ec4c9f9f17 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.099463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 420/830] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:08.105919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.108564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.108564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.128749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:08.130951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:08.135700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5526d-ab4a-4cdc-8043-7e0b066c9d5c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.167572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.175669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.176402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.183050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.183050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.184624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:08.186265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:08.199539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:08.200655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:08.201545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 411/831] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:08.206835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.226735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 421/832] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:08.332886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786dcda8-4f3d-4213-b41c-11238565b51d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.369121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.369986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.370416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.370864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.371714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:08.372167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:08.373794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de3a25-f613-4d84-96e3-8b264603b1f7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.387269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 412/833] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:08.396380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.431855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.445127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6b0d-c004-465d-bfd3-c55738fa78c2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.445627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6b0d-c004-465d-bfd3-c55738fa78c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.446077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e6b0d-c004-465d-bfd3-c55738fa78c2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.449512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 413/834] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:08.454679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.454895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.455169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.474062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:08.475253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:08.522349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9641-f007-48a2-85e0-c6636e87e3f5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.537535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.563488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 414/835] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:08.563852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.564309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.564822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.565240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.565660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.566943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=4542cf167cc24738b441af6530dead3c) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:08.567391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:08.570396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.583288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 422/836] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/role_assignments?user.id=4542cf167cc24738b441af6530dead3c&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:08.682439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3bacb-83ca-4457-9435-9e98fff0cc8a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.723230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d19d9-2706-4355-b582-92373e3156df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.723795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d19d9-2706-4355-b582-92373e3156df None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.724072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d19d9-2706-4355-b582-92373e3156df None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.724369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d19d9-2706-4355-b582-92373e3156df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.737074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d19d9-2706-4355-b582-92373e3156df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-005d19d9-2706-4355-b582-92373e3156df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:08.741849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:08.743775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:08.744646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:08.748459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 415/837] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:08.750173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.839023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d19d9-2706-4355-b582-92373e3156df None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.839927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.860735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c9539-c48b-451e-835a-7ffa04e5741e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.881436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:08.903379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.904257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.904564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.904842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.905509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:08.905810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:08.907170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9e647-8193-43c1-b0da-f319bf0f8c3d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:08.920649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 423/838] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:08.920649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221089c3-e748-4bd3-b6ab-fe14b9fce8f0 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.920649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221089c3-e748-4bd3-b6ab-fe14b9fce8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.920649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221089c3-e748-4bd3-b6ab-fe14b9fce8f0 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:08.922138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 424/839] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:08.925370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221089c3-e748-4bd3-b6ab-fe14b9fce8f0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:08.958448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d19d9-2706-4355-b582-92373e3156df None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:08.994710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6532a8-3c83-441d-a763-761510fa744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:08.995334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6532a8-3c83-441d-a763-761510fa744f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:08.995671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6532a8-3c83-441d-a763-761510fa744f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:08.995939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6532a8-3c83-441d-a763-761510fa744f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.006479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6532a8-3c83-441d-a763-761510fa744f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-be6532a8-3c83-441d-a763-761510fa744f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:09.011360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:09.012306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:09.013207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 416/840] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:09.014396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.014396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.014396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.032532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:09.033708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:09.043851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221089c3-e748-4bd3-b6ab-fe14b9fce8f0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.084601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.085659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.085949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.086462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.098187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.099634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.105067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 417/841] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:09.106160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:09.115114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:09.116321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:09.117274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:09.117274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:09.117274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:09.121685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.122680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 425/842] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:09.127965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.149011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.241492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225c7fd-c710-4632-bc85-789cb2fe5217 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.245085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42853f-27c3-46f4-812d-5946d56401b2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.280871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.281895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.282180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.282454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.283435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:09.283747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.287460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.292845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.293760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.294253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.294733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.295599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:09.296094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.297817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.312046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 418/843] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.312673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 426/844] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.313149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.314853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.379362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.408047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.418929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c98e3-68a3-49d8-bb35-e6d7378f7f08 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.429048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861d751-0430-478b-8a6c-ee55ac870ae4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.455957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.456622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.456880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.457145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.465442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.466917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.467914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.468561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.468838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.469106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:09.480276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:09.481315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 16:26:09.487335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:09.487871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.487944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 419/845] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:09.488024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.488093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:09.501554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:09.502652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:09.503545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:09.503545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:09.503545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:09.503545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 427/846] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:09.503545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.597151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11a4f0-c2f4-4001-8a29-f792d83a6b1e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.613682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e88536-7a39-48fd-8350-8448431caf22 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.625381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.632187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.636214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.636214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.636214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.636214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.636214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:09.645044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:09.646019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:09.646879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 420/847] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:09.647679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.655959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.657054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.657528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.657752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.658684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:09.659369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.660878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.665441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.683130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 428/848] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.684052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.756548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396741bc-25a4-4c4c-8728-2f8d5be082b7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.793177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed9a0f-68dc-44bf-bb1a-142db3b3e4e6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.800195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.800851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.801129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.801666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.802040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:09.802341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.803732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.818926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 421/849] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.824239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.836232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.837031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.837317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.837618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:09.838209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:09.838510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:09.840092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:09.849051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 429/850] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:09.849051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:09.882955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.908046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:09.931696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7743b1-ede9-4855-b0b3-2c61d37bd523 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.971280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b98d8d-b82f-489a-8b64-5680ab684d32 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:09.975311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:09.975924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:09.976196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:09.976496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.011032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.011586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.011869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.012137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.020266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.021728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:10.034864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:10.035926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 430/851] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:10.037629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.064073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.065576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.139302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.148539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499165a3-e41b-456e-a5a2-a68a9e000463 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.151210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.177031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 422/852] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:10.177861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.189595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.190192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.190468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.190736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.191364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:10.191665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.193053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.204570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 431/853] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:10.204570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.292412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cc24f-eef2-4f55-aabc-66be6b38c8c2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.335799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.337244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.337735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.338114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.338877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:10.339299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.340927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.345447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e21a-387a-47bc-bbe9-b15924dff42b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.372813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 423/854] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:10.376011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.392629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.393291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.393569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.393839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.408691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.410361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.424369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:10.430150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:10.437456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:10.438355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:10.438355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:10.438355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:10.438355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 432/855] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:10.439476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.454873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.500548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d472f50-91ca-4c02-a9f5-42f1d4c77b5a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.548193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567e14e-cce8-43bc-8c40-03e2e63c954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.549323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567e14e-cce8-43bc-8c40-03e2e63c954c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.550218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567e14e-cce8-43bc-8c40-03e2e63c954c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.550639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567e14e-cce8-43bc-8c40-03e2e63c954c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.576266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626d134-35c8-4194-bc03-43d6143760c0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.630840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.632741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.633414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.635147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.636492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:10.637291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.640518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.667200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567e14e-cce8-43bc-8c40-03e2e63c954c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.667200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567e14e-cce8-43bc-8c40-03e2e63c954c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.668032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 433/856] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:10.670518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.705709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.768597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.787538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 424/857] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:26:10.787538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4c94b-60b9-422c-b9a4-894410f879fc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.787538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4c94b-60b9-422c-b9a4-894410f879fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.787538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4c94b-60b9-422c-b9a4-894410f879fc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.787538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 425/858] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:10.792901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4c94b-60b9-422c-b9a4-894410f879fc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:10.821914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa754a-493d-4eeb-9e5a-9dce1f0e1656 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.865292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.865920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.866208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.866461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.873919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.875355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6111437d-15cd-4c4f-a0d3-26b17c3a55d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:10.886853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:10.887780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:10.888696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 434/859] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:10.889679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.889786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.890000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.912582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:10.913764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:10.925088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4c94b-60b9-422c-b9a4-894410f879fc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:10.966430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:10.972873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:10.973541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:10.973820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:10.994510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:10.994510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:10.994510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.000509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 435/860] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:11.001458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:11.026106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:11.027881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:11.029732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:11.029732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:11.029732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:11.031910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 426/861] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:11.037260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.106274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.150109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69529d7c-dd23-4d69-b2bd-fee4a0b70333 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.186042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabc938-3567-42ef-90dd-9973309d95d4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.189136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.189984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.190457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.190954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.191850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.192381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.194117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.212800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 436/862] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.216529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.225253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.225937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.226216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.226485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.227209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.227565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.229586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.252044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 427/863] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.253268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.307990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.329924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0eb8f-b355-434a-8dfe-c0d7df8c8484 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.350083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.368413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d5857-8a45-47fa-a317-936517ff67d8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.368718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.369305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.369581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.369863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.370772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=1f762db26df84f5faf7e6778383e2870) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.371086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.372591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.385246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 437/864] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/role_assignments?user.id=1f762db26df84f5faf7e6778383e2870&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.386494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.413231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.415107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.415600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.416118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.427262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.429546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:11.447676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:11.448703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:11.449587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:11.452876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 428/865] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:11.458492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.516630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b768e-1900-483e-980b-e6b9479d71ec None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.561530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.562164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.562438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.562704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.563506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.564923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:11.579759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:11.580804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:11.581748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:11.582482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 438/866] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:11.587827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.589962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaffb1-44ac-4f98-a7b3-6934f9e745d5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.595039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.640426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.641780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.642273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.642971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.643846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.644621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.646319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.646929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.666019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 429/867] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:11.672504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.709255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70427558-e6d3-47e5-a190-9cc93ab71ccc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.745568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.746222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.746495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.746802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.747432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.747727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.749746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.754568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 439/868] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:11.760929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:11.786603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902993b-b91c-44b2-99a2-a40807af9106 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.826860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.828420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.828918 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.829386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.830341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=270fe102379a4346ae009a4545865c28) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.831111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.833353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b109e88-c3dc-4fab-9230-55724d55d392 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.840455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 430/869] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/role_assignments?user.id=270fe102379a4346ae009a4545865c28&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:11.876882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bada0a-4fc5-45b7-9721-31ac2b5f1bd9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:11.892990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:11.923372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:11.924555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:11.925012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:11.925491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:11.926306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:11.926778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:11.928433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640a9d0-4a00-4cb5-b86d-fb19de503ba0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:11.934892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 440/870] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:12.237034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187d0b4-477b-4a71-838b-9c5c3ec0f499 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.237221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187d0b4-477b-4a71-838b-9c5c3ec0f499 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.237495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187d0b4-477b-4a71-838b-9c5c3ec0f499 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.239130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 431/871] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:12.245267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436cd9a-3877-4e21-91a5-d9ba4a0857e1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.245551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436cd9a-3877-4e21-91a5-d9ba4a0857e1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.245823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436cd9a-3877-4e21-91a5-d9ba4a0857e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.264890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4436cd9a-3877-4e21-91a5-d9ba4a0857e1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:12.266068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4436cd9a-3877-4e21-91a5-d9ba4a0857e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:12.337720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 441/872] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:12.611950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187d0b4-477b-4a71-838b-9c5c3ec0f499 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:12.640965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:12.717295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187d0b4-477b-4a71-838b-9c5c3ec0f499 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:12.750569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f56f5-e81e-4d37-858d-09e358c83c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:12.751175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f56f5-e81e-4d37-858d-09e358c83c28 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:12.751448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f56f5-e81e-4d37-858d-09e358c83c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:12.751717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f56f5-e81e-4d37-858d-09e358c83c28 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:12.752447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f56f5-e81e-4d37-858d-09e358c83c28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:12.753869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f56f5-e81e-4d37-858d-09e358c83c28 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:12.828201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 432/873] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:26:12 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:13.441806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141783-0a02-416c-bb47-15d070415949 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.442355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141783-0a02-416c-bb47-15d070415949 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.442355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141783-0a02-416c-bb47-15d070415949 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.443934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 442/874] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:13.449703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35cb00-02f1-42ec-a0c2-8c3dd74f369a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.449992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35cb00-02f1-42ec-a0c2-8c3dd74f369a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.450269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35cb00-02f1-42ec-a0c2-8c3dd74f369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.469988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1e35cb00-02f1-42ec-a0c2-8c3dd74f369a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:13.471210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1e35cb00-02f1-42ec-a0c2-8c3dd74f369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:13.549994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 433/875] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:13.819201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141783-0a02-416c-bb47-15d070415949 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:13.875757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d593ed-63f9-4c8e-9e55-abbe21ffd32a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.876303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d593ed-63f9-4c8e-9e55-abbe21ffd32a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.876303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d593ed-63f9-4c8e-9e55-abbe21ffd32a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.877886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 434/876] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:13.882813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf5541-acce-4c93-aaf5-7fdde05011da None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.883094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf5541-acce-4c93-aaf5-7fdde05011da None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.883384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf5541-acce-4c93-aaf5-7fdde05011da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.904794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-72bf5541-acce-4c93-aaf5-7fdde05011da None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:13.906240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-72bf5541-acce-4c93-aaf5-7fdde05011da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:13.917143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:13.939794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141783-0a02-416c-bb47-15d070415949 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:13.942235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:13.978472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:13.979099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.979434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.979695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.980301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:13.981688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:13.984494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 435/877] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:13.985779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11ab2c-fa79-455a-947e-abd812896a63 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.986145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11ab2c-fa79-455a-947e-abd812896a63 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.986310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11ab2c-fa79-455a-947e-abd812896a63 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.987793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 436/878] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:13.993162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:13.993438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:13.993707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:13.997432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:13.998424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:13.999261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:14.003578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 443/879] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:14.005359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.016051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:14.017557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:14.092801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 437/880] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:14.124940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89ef74-3df6-4b41-ad3e-813e337984a3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.161484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.162091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.162369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.162641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.163260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:14.163558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.165094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.169920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 444/881] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:14.175363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.251635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.254900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.286428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37064c-00a6-4851-8f04-6cdc490842d8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.290432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.331368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.331983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.332545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.332823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.333471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:14.333771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.340579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.348472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 438/882] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:14.373151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.376150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ff487-68ad-43c5-bf10-9038705ca8c9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.419701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.420366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.420648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.420923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.436939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:14.443636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:14.445563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:14.446527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:14.448426 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 445/883] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:14.451407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.511950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af744b96-19be-4b7f-bde5-08274af04b33 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.557178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e2adc-6709-4871-b146-48675e12196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.558766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e2adc-6709-4871-b146-48675e12196a None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.559267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e2adc-6709-4871-b146-48675e12196a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.559731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e2adc-6709-4871-b146-48675e12196a None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.561084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e2adc-6709-4871-b146-48675e12196a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.563477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e2adc-6709-4871-b146-48675e12196a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.577039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.585519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0e38e-5dbe-43a2-a29a-e07159332010 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.623604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.624191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.624500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.624772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.625431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:14.625725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.627075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.636135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 446/884] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:14.641342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.671517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.683595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 439/885] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:26:14 2022] POST /identity/v3/services => generated 269 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:26:14.760390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff917-81b5-4094-a57c-73c70e78067a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:14.806893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:14.807810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.808238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.808694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.821605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:14.823319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:14.831105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1b228-d62e-4dec-b484-95850398b877 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.831105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1b228-d62e-4dec-b484-95850398b877 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.831435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1b228-d62e-4dec-b484-95850398b877 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.836549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 440/886] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:14.839065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:14.839353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:14.839629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:14.840317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:14.841254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:14.842763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:14.842763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:14.842763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:14.842763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 447/887] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:14.847248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:14.850859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:14.863638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:14.864913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:14.952458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 441/888] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:14.976691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc8a02-feaf-414c-ae26-d85d517f8359 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.012068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.012725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.013051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.013394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.014045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.014345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.015723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.029298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 448/889] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:15.037458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.147725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f74761-5143-40cf-b289-9a51078c8103 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.177321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.186713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.190516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.190516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.190516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.196498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.198277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:15.210324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:15.211425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:15.212365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:15.214544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 442/890] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:15.220431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.246221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.291657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.346905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fd2d1-ee46-4c1c-918b-c031ea6fcc40 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.366529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bb36-834c-4e40-b01d-aa1bb21639a1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.387015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.387627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.387904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.388171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.388934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.389350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.390778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.403962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.404984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.405471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.405942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.406761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.408394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.410712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 449/891] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:15.416398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:15.418871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:15.419831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:15.420729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:15.423121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 443/892] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:15.430077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.525719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bce66-6fb2-479b-ab50-59b4d2825895 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.534120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.548888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.563074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.563988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.564519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.564991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.566464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8626-380a-472a-8b22-549b8d681e9d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.608862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.609901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.610837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.612064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.612880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.613696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.615444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.622563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 444/893] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:15.628312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.646557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.648018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.763543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd9b2-0814-402f-b67b-1099a1856cd7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.777549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 450/894] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:26:15.783540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.786756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.806482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.807090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.807362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.807628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.808258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:15.808584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.808862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:15.809974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:15.819569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 445/895] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:15.821162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:15.921467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69fec0-1ba3-44e5-907e-e7f2670d57d3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.962437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649d30-a23a-40a5-a835-931bdeb4368e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:15.968204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:15.968964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:15.969335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:15.969722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:15.983757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:15.985973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:16.003097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:16.004309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:16.005425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:16.005425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:16.005425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:16.008442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.012247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.012247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.012247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.012247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] RBAC: Adding query filter params (interface=public, service_id=46294255b8ba445485edd7138b58f69d, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:16.012623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.012750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 451/896] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:16.014182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.014586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.024894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 446/897] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:15 2022] GET /identity/v3/endpoints?service_id=46294255b8ba445485edd7138b58f69d&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:16.026472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.088308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.107826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.146267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f760cf-aa77-443b-a21d-b61d83c8b3d7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.157530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7652495-8bca-4047-8270-1368bed21630 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.189291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.189904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.190184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.190451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.191243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:16.191549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.193592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.202281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264795b0-8dc9-461b-8524-75003cf10e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.203248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264795b0-8dc9-461b-8524-75003cf10e84 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.203729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264795b0-8dc9-461b-8524-75003cf10e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.204200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264795b0-8dc9-461b-8524-75003cf10e84 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.205441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264795b0-8dc9-461b-8524-75003cf10e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.207290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264795b0-8dc9-461b-8524-75003cf10e84 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.212398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 452/898] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:16.214682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.217585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 447/899] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/services => generated 1268 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:26:16.331395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe499e1-a650-4086-b734-d708ad77d690 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.356031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585796d-1c8b-4338-8c76-5189d693c4e0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.356489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585796d-1c8b-4338-8c76-5189d693c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.356858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585796d-1c8b-4338-8c76-5189d693c4e0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.358667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 448/900] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:16.365814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.367010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.368174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.368741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.371438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.372163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.372493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.372828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.381205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.382316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.392139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:16.393373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:16.394150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:16.395020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:16.395899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:16.400581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 453/901] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:16.401562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.482084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 449/902] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:16.518630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074e49-bfb3-4901-9988-d59deec032c9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.553853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.554520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.555434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.555763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.556596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:16.556902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.558324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.576439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 454/903] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:16.578658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.623246 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.696547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d0d85-2326-4b11-a287-a575839d323f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.743188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.744145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.744663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.745111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.746101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] RBAC: Adding query filter params (role.id=adc0f1e2c62f4960b6cff713d64e608f, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=270fe102379a4346ae009a4545865c28) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:16.746580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.748260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c21154-dfa7-45b6-a934-8fc5e958b0cb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:16.760527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 450/904] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:16 2022] GET /identity/v3/role_assignments?user.id=270fe102379a4346ae009a4545865c28&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=adc0f1e2c62f4960b6cff713d64e608f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:16.783776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:16.797534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c647-c3b3-4eb9-9c84-1278521f7589 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.797849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c647-c3b3-4eb9-9c84-1278521f7589 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.798072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c647-c3b3-4eb9-9c84-1278521f7589 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.799677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 451/905] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:16.806774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.807084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.807496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.831292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:16.833615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:16.860761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:16.922935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d99fe-cf3b-4565-b681-72c118a41025 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:16.936400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 452/906] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:16.959557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:16.960174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:16.960484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:16.961302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:16.961956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:16.963389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.036946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 455/907] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:26:16 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:17.204754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.237815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:17.312727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae9b3-074e-4915-bbbe-58f23237cac1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.356739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.357775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.358254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.358742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.359519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:17.361157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:17.371418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:17.372384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:17.373297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:17.376009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 453/908] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:17.382102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.495918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3688130-660d-4db7-a98d-68ce42a7dc6a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.509623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:17.531962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4052-1186-464b-918e-07060968b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.532607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4052-1186-464b-918e-07060968b473 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.532890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4052-1186-464b-918e-07060968b473 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.533162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c4052-1186-464b-918e-07060968b473 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.533792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c4052-1186-464b-918e-07060968b473 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:17.534093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c4052-1186-464b-918e-07060968b473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:17.535455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c4052-1186-464b-918e-07060968b473 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.541803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 456/909] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:17.547420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.647148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:17.656838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ef5da-33da-4c4e-b8a3-8273b99cacf4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.691842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.692485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.692767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.693040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.693737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:17.694040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:17.695839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.701313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 454/910] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:17.707045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4052-1186-464b-918e-07060968b473 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.825691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c4052-1186-464b-918e-07060968b473 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:17.861685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:17.862296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:17.862571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:17.862842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:17.863545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] RBAC: Adding query filter params (interface=public, service_id=08483de2a6e540b7bc4a86fef5cc8790, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:17.863853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:17.865246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1492-6cdd-4478-8920-0bf071829d4c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:17.871524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 457/911] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/endpoints?service_id=08483de2a6e540b7bc4a86fef5cc8790&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:17.876991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:17.997404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76818-c993-467b-a4f9-5b145c024065 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.033913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c263f1-a620-4351-bf30-2904f9c56aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.034902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c263f1-a620-4351-bf30-2904f9c56aca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.035403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c263f1-a620-4351-bf30-2904f9c56aca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.035890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c263f1-a620-4351-bf30-2904f9c56aca None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.036724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c263f1-a620-4351-bf30-2904f9c56aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.038308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c263f1-a620-4351-bf30-2904f9c56aca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.045732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 455/912] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:17 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:26:18.245423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02183850-bec1-4b13-b286-13352d789453 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.245731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02183850-bec1-4b13-b286-13352d789453 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.246107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02183850-bec1-4b13-b286-13352d789453 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.247700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 458/913] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:18.252913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.253370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.253449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.272801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:18.274101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:18.288765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:18.346408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 456/914] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:18.617571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02183850-bec1-4b13-b286-13352d789453 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.665124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:18.735697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02183850-bec1-4b13-b286-13352d789453 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.771518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.772154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.772452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.772725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.773354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.774801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-78caf97b-3127-4405-a21d-b8941e44b2a9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:18.784168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:18.785204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:18.786044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 459/915] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:18.792037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.833871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768e1f-49bf-4a4e-9cc3-d7c79ac0f93c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.834495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768e1f-49bf-4a4e-9cc3-d7c79ac0f93c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.834970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768e1f-49bf-4a4e-9cc3-d7c79ac0f93c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.836971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 460/916] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:18.842097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.842636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.843110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.863294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:18.864990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:18.904788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d8eda-08a9-4294-b7c5-9ba007d28c30 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:18.948383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:18.948994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:18.949268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:18.949532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:18.950173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:18.950472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:18.951873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:18.961039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 461/917] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:18.961187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 457/918] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:18.965644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:18.998892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.069452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68439f-6faa-4b04-aaf3-1ceb5e29b877 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.105527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.106132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.106411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.106730 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.107364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:19.107659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:19.109048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c3cb7-453c-4907-ab06-8643600315f8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:19.114574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 462/919] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Aug 24 16:26:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:19.121328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.121764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78317cf-16f0-4ec1-a9b1-da0d0fe012c7 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.122043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78317cf-16f0-4ec1-a9b1-da0d0fe012c7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.122310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78317cf-16f0-4ec1-a9b1-da0d0fe012c7 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.123834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 463/920] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:19.129626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.129904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.130176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.139735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.151189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:19.152872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:19.243272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0582c40-5016-47ee-84f5-6f801078ea66 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.247425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 464/921] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:19.248082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.277341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.301295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.302580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.303486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.304091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.305150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:19.307460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:19.352561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4f957-f4ea-4cc0-b1d7-46626c29080f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.388420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.389054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.389409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.389606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.395596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 458/922] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:19.408454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:19.410821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:19.411797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:19.412666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 465/923] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:19.420499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.505996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.545283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.548783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77a994-8bab-437e-8775-ed099eccf8aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.591460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.592523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.593004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.593465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.594467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:19.594961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:19.596851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:19.607948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 459/924] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:19.613765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.669096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.684998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d0b0b-85b7-4ad4-85bc-c6784b9c783d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.730523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.731162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.731442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.731713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.732302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:19.734465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:19.746925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:19.748449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:19.749284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 466/925] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:19.756454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.763978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af610024-2043-4f56-b861-45dad2ddb90d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.800517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.810965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.816552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.816552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.816552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.824972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:19.826742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:19.843231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:19.844202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:19.845120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:19.845120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:19.845120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:19.852523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 460/926] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:19.853699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.878598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9d3f2-01ab-4637-aa16-5551dac42d3f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:19.928320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:19.929788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:19.929788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:19.930247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:19.930975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:19.931275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:19.932718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:19.942514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 467/927] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:19.943533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:19.964119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:19.969775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69397c-7ac4-464c-a2f0-7f5f5a5873f2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.007077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.007872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.008426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.008830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.009595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:20.010042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.011561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.028544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 461/928] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:20.031254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.053350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e402d-c99a-4c6c-9e39-6f6551864b11 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.078751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.093477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.094414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.094697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.094998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.095609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:20.095908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.097347 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394e2fc-e9d3-49c7-bac0-53666c778c39 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.102910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 468/929] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:20.104158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce091e-83f4-4fd3-ace7-d89503d6aedf None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.104499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce091e-83f4-4fd3-ace7-d89503d6aedf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.104781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce091e-83f4-4fd3-ace7-d89503d6aedf None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.106147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 469/930] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:20.108648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce091e-83f4-4fd3-ace7-d89503d6aedf None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.150042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6fa69-d792-4069-9af8-91c49e642f84 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.184096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.184677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.184950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.185208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.192083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.193575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fff9e543-a3a9-41d3-8558-2f0e3ddd4f42 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:20.205342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:20.206275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:20.207203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 462/931] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:20.207796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.207796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.208551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.214097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.227414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:20.230215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:20.234054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce091e-83f4-4fd3-ace7-d89503d6aedf None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.270840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.271531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.271811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.272078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.272862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] RBAC: Adding query filter params (interface=public, service_id=36a2c83c24cc4ba6ab9da8a99fa5116b, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:20.273695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.275076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.280422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 470/932] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/endpoints?service_id=36a2c83c24cc4ba6ab9da8a99fa5116b&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:20.281190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.302661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 463/933] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:20.303559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.336928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.406914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee4907-03f5-4830-94b7-8d41e4365f5e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.420704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284e21d-565c-470f-9657-64f92a037a50 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.443928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.444617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.445458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.445817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.446629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:20.446854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.448331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.467351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.467351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.467351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.467351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.467761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.467761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.471041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 471/934] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:20.471927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.473727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 464/935] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:26:20.474968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.573945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.603986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b897f1-3138-4d70-a777-86c08bf96f3d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.653045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.653947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.654223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.654494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.666783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.668590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:20.686000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:20.686987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:20.687915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:20.687915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:20.687915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:20.687915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 472/936] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:20.693386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.698224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.703696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ef48-5213-49f8-8628-8e1aa6f31a45 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.742498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.743235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.743517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.743784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.744389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.746061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:20.755782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:20.756862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:20.757782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 465/937] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:20.764737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.766055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:20.817126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2025-d9e7-4311-a683-0dce87aca130 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.859694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.860308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.860616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.860886 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.861531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:20.861830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.863199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.876994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 473/938] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:20.882884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.897633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1078118-9c13-498e-b870-6a861c858e07 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:20.948027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:20.949359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:20.950015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:20.950488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:20.951913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:20.952509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:20.954891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:20.962327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 466/939] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:20.967619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:20.994499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.003959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59202f-0623-4c10-bc80-bda57a7e158b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.025300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.048654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.049438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.049748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.050099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.058268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.059889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:21.072113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:21.073162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:21.074154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:21.076858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 474/940] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:21.079402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.105045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a0acb-f1fa-462c-a996-93cb8031e5e3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.151828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.152555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.152909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.153454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.154400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:21.154699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.156125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.166931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 467/941] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:21.167916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.200724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8755c-fa82-46b6-a1e4-b07fdb946c4f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.238046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.238547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.238887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.239223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.239974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:21.240410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.241957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.259661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 475/942] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:21.260707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.284997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.297752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.307669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8ad64-79b4-489f-a52b-88136dcaf091 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.353655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebef4b4-e861-47de-b5b0-cdcc3edd9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.354354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebef4b4-e861-47de-b5b0-cdcc3edd9fb0 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.354701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebef4b4-e861-47de-b5b0-cdcc3edd9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.355041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebef4b4-e861-47de-b5b0-cdcc3edd9fb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.356038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebef4b4-e861-47de-b5b0-cdcc3edd9fb0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.357922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebef4b4-e861-47de-b5b0-cdcc3edd9fb0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.374576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de5fff-4c50-432c-a956-f3ebcbaffbae None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:21.410630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:21.411389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.411667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.411935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/270fe102379a4346ae009a4545865c28/roles/adc0f1e2c62f4960b6cff713d64e608f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.414372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=270fe102379a4346ae009a4545865c28, role_id=adc0f1e2c62f4960b6cff713d64e608f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:21.415858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:21.458410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 468/943] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:21.463433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec132-38ea-43fc-9eed-25bd6d9f8234 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.463741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec132-38ea-43fc-9eed-25bd6d9f8234 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.463998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec132-38ea-43fc-9eed-25bd6d9f8234 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.465729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 469/944] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:21.471592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:21.471907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:21.472137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:21.492632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:21.493856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:21.512113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 476/945] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:21 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/270fe102379a4346ae009a4545865c28/roles/adc0f1e2c62f4960b6cff713d64e608f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:21.589497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 470/946] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:21.864902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:21.974704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:21.981923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ef32b-dccf-4335-b667-af70fff3edbc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.019804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.020895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.021436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.021915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.022806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.024657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.027491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING keystone.server.flask.common [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 16:26:22.078828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.sql.core [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:26:22.078828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 16:26:22.078828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [parameters: {'id': '19b2a712623045d3b29e932fdcf82169', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 16:26:22.078828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81626) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19b2a712623045d3b29e932fdcf82169.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19b2a712623045d3b29e932fdcf82169. Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:26:22.087695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 16:26:22.095253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application yield resource Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise self.value Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application session.commit() Aug 24 16:26:22.096249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:26:22.097719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:26:22.098630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:26:22.099424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [parameters: {'id': '19b2a712623045d3b29e932fdcf82169', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.100288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:22.101715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 19b2a712623045d3b29e932fdcf82169. Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:22.103685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 477/947] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Aug 24 16:26:21 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:22.104876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.176642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.216836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54545f11-835a-475a-af13-1f99eaedbfac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.251892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.252561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.252907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.253325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.261317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.262785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-57c80ae3-93fc-4f0b-be80-194bce4fee5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:22.274253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:22.276386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:22.277356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 471/948] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:22.282832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:22.388869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ed51-eb1d-4f3c-ae6d-2ddccb16ef50 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:22.425419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:22.426401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.426955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.427437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.428313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:22.428854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:22.431285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3e171-aae0-435a-9886-cf63f549f8a6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:22.452401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 478/949] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:22.530335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cfade-4618-4e1c-81a9-a848ba84581c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.530893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cfade-4618-4e1c-81a9-a848ba84581c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.530893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cfade-4618-4e1c-81a9-a848ba84581c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.532634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 472/950] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:22.538967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:22.539354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:22.539565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:22.559998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:22.561629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:22.566293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:22.638096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 479/951] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:22.912679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cfade-4618-4e1c-81a9-a848ba84581c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.022039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cfade-4618-4e1c-81a9-a848ba84581c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.053531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.061020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.061638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.061916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.062213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.062865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.064424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:23.078591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:23.079786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:23.080716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 473/952] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:23.081551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.190218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9249a-4557-4048-bc6c-fdbfd48686ba None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.226681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.227299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.227576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.227849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.228511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:23.228814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.230169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.235418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 480/953] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.240723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.349919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd400d-152c-49a1-a310-d15800acd20f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.386247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.386865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.387142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.387413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.388054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:23.388364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.389769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc31a-5d5b-4c32-8587-cf6e1567c563 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.395256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 474/954] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:23.401593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:23.413144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.514071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8da08-d8f6-49a0-9907-ca7bb3c586bd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:23.531288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ee023-7367-494a-9624-a8ef67218f58 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.531823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ee023-7367-494a-9624-a8ef67218f58 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.532108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ee023-7367-494a-9624-a8ef67218f58 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.534606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 475/955] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:23.541709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.541995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.542627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.555608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50719b4-875e-49be-ad93-708755cf2f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:23.556610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50719b4-875e-49be-ad93-708755cf2f3f None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.557106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50719b4-875e-49be-ad93-708755cf2f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.557708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50719b4-875e-49be-ad93-708755cf2f3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.558384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50719b4-875e-49be-ad93-708755cf2f3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:23.559732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50719b4-875e-49be-ad93-708755cf2f3f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:23.563635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:23.565796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:23.622111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.630634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 481/956] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:23 2022] POST /identity/v3/endpoints => generated 349 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:23.632248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986a2d-00f7-42b0-af97-b36fcdd60a83 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.632938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986a2d-00f7-42b0-af97-b36fcdd60a83 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.633213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69986a2d-00f7-42b0-af97-b36fcdd60a83 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.636393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 482/957] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:23.640772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:23.641291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:23.641753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:23.668306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 476/958] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:23.668592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:23.668592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:23.739514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:23.753321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 483/959] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:23.947895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.041103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.050482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655a470-d083-4eac-98d8-83bede5b81ad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.093278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.093916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.094188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.094457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.095114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.097236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:24.106472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:24.109615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:24.110466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:24.112710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 477/960] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:24.113992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.160020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87d3f-5d02-41f8-992f-101f97bf0a73 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.167359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.199740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803596-d400-4301-939e-5bce769332ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.201065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803596-d400-4301-939e-5bce769332ae None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.201974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803596-d400-4301-939e-5bce769332ae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.202655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a803596-d400-4301-939e-5bce769332ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.216609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a803596-d400-4301-939e-5bce769332ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6a803596-d400-4301-939e-5bce769332ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:24.222078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:24.224135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:24.225488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:24.227030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a3d0d-695e-4dd9-8552-0f8a48e15a54 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.229465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 484/961] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:24.237334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803596-d400-4301-939e-5bce769332ae None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.274553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.275697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.276177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.277079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.278559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.279062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.281742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.287919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 478/962] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:24.295523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.304864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.365998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a803596-d400-4301-939e-5bce769332ae None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.405323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.406281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.406756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.407248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.408220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.408734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.410478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.413784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbde634-9b18-4105-bd6c-6e6851402924 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.420312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 485/963] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:24.426180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.429451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.461020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.462824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.463498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.464212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.465042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.465766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.467965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dec4d-a3ff-49ad-922c-0f4255e3ea9e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.474247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 479/964] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:24.536786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a8a1f-e615-413a-b3fd-4586b690c0c3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.539158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a8a1f-e615-413a-b3fd-4586b690c0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a8a1f-e615-413a-b3fd-4586b690c0c3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.542409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 480/965] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:24.548253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.548606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.548878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.561802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8374d-37e0-43ef-868d-11842871f1f4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.578426 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.579365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:24.580564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:24.605667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.606663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.607112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.607549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.619575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.621357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:24.640899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:24.641932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:24.643041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:24.643041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:24.643041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:24.643041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 486/966] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:24.649789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.680524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 481/967] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:24.771980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80753ab-3234-4b1d-b7c9-4a5105a03f2e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.809485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.810087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.810362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.810625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:24.811258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:24.811557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:24.812943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:24.825842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 487/968] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:24.833793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.951498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857402d-03b8-428b-849f-79493d3a9795 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:24.954296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.959244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:24.988808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:24.990511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:24.991603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:24.992499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:24.994167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.003807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.006209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:25.023180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:25.024491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 16:26:25.025437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:25.026698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 482/969] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:26:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:25.035273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.066550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f42e2-e39c-431e-8af4-d290f58699f6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.105348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.105961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.106238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.106504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.107133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.108708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.154753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.sql.core [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:26:25.154753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 16:26:25.154753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [parameters: {'user_id': '981aa17f417e42818c1619b76f9158c9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 16:26:25.154753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81626) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 16:26:25.161596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e2d2-79f9-4b88-a3d9-db9e09bc0206 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 24 16:26:25.162593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:26:25.164495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application yield resource Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise self.value Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application session.commit() Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 16:26:25.165535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.flush() Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application compat.raise_( Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 16:26:25.166498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 16:26:25.167378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 16:26:25.168476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [parameters: {'user_id': '981aa17f417e42818c1619b76f9158c9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:25.169348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 16:26:25.170166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 488/970] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Aug 24 16:26:24 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:25.171120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.217848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.218775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.219199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.219620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.220519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:25.220965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.223165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.253615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 483/971] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:25.254903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.285923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.292653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.302090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bf68b-0d2f-409a-8f13-3dcbb24d3ebe None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.339167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.339887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.340166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.340561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.349560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.351744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:25.365242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:25.366933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 16:26:25.367880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.372520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 489/972] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:25.373168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.397752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c545fe-0694-4a13-bb8c-ffe2471ee966 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.448591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885c4d4-3993-4106-aab5-99eb72bad0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.449599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885c4d4-3993-4106-aab5-99eb72bad0bd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.450342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885c4d4-3993-4106-aab5-99eb72bad0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.450814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885c4d4-3993-4106-aab5-99eb72bad0bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.487065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf4eea-5307-4285-9654-17091931bc3b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.523793 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.524505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.524791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.525095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.525762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:25.526062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.527493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.542854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885c4d4-3993-4106-aab5-99eb72bad0bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.544323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885c4d4-3993-4106-aab5-99eb72bad0bd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.570204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.573696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 490/973] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:25.667857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 484/974] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:26:25.673297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.678321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.703922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78d1d-06e5-4f7b-80e1-13fd5865fd83 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.704324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78d1d-06e5-4f7b-80e1-13fd5865fd83 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.704630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78d1d-06e5-4f7b-80e1-13fd5865fd83 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.706472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 485/975] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:25.716849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.717129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.717403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.736727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:25.737996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:25.805159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55ef63-d05d-45b2-a0cc-f7b427bd3996 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:25.831184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 486/976] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:25.847609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:25.848532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:25.848987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:25.849430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:25.862970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:25.862970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:25.878463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:25.879471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:25.880417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:25.880417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:25.880417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:25.880417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 491/977] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:25.887299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:25.942216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:25.991542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c8e08-9b2d-447c-80ef-f803778f378a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.028588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.029193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.029467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.029734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.030371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.030667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.032048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.045149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 487/978] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.051122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.109581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.146517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.155106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b603d1-99ea-4cf6-b5e2-84b1dc7b8d59 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.192129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.192834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.193046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.193306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.200870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.202317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:26.214633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:26.215557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:26.216554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:26.220591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 492/979] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:26.221406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.262175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6679-b68f-41fa-9918-3c2160b347c0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.276955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.300663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.301292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.301569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.301838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.302419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.304622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:26.314005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:26.314973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:26.315837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 488/980] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:26.322235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.337081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0ebf-5b6a-4b2e-9f53-5a5fd2d100c4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.357246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.376329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.376966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.377245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.377517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.378212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.378518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.379953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.405296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 493/981] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.406141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.442161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e2241-223a-4130-a805-1a8ff43ae704 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.478346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.478950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.479225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.479490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.480118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.480433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.481817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.487421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 489/982] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:26.493549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.514905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70b4f8b-969f-4421-a870-ad0e2d7af998 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.540421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.551672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.552494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.552943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.553356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.554751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] RBAC: Adding query filter params (role.id=adc0f1e2c62f4960b6cff713d64e608f, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=270fe102379a4346ae009a4545865c28) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.555219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.556809 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a895513-d046-4e7b-af1f-6bea752f051c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.562908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 494/983] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/role_assignments?user.id=270fe102379a4346ae009a4545865c28&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=adc0f1e2c62f4960b6cff713d64e608f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:26.606000 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb3023-0d6c-4e56-bc44-0cdd67780843 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.623129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a569-f632-4d77-9a21-35ac15248388 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.623409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a569-f632-4d77-9a21-35ac15248388 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.623690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a569-f632-4d77-9a21-35ac15248388 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.625102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 495/984] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:26.630680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.630958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.631234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.652665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.652933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:26.653952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.654369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.654556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:26.654973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.655763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.656232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.659217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.668514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 490/985] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:26.672567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.703665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.754570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 496/986] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:26.806813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91b2da-47ab-4ff6-80bf-9ad23fe78018 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.810249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:26.840901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.841601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.841949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.842292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:26.843224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] RBAC: Adding query filter params (interface=admin, service_id=36a2c83c24cc4ba6ab9da8a99fa5116b, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:26.843596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:26.845234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:26.856087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 491/987] 10.222.0.17 () {56 vars in 1249 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/endpoints?service_id=36a2c83c24cc4ba6ab9da8a99fa5116b&interface=admin®ion_id=RegionOne => generated 193 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:26.857355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:26.964661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cc4eb-3bf1-4e09-81d5-07329e600628 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:26.998747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1bee1-ff72-4b45-8f3c-831786ddc8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:26.999383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1bee1-ff72-4b45-8f3c-831786ddc8ec None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:26.999660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1bee1-ff72-4b45-8f3c-831786ddc8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:26.999929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1bee1-ff72-4b45-8f3c-831786ddc8ec None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.000582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1bee1-ff72-4b45-8f3c-831786ddc8ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:27.001936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1bee1-ff72-4b45-8f3c-831786ddc8ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:27.008978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 497/988] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:26 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:26:27.032505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:27.153778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267cb-db2e-4002-a92d-a024635d380f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:27.190250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab27ed-a917-4cd7-b4b4-a6b102eddf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:27.190855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab27ed-a917-4cd7-b4b4-a6b102eddf2e None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:27.191130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab27ed-a917-4cd7-b4b4-a6b102eddf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:27.191392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab27ed-a917-4cd7-b4b4-a6b102eddf2e None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:27.192031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab27ed-a917-4cd7-b4b4-a6b102eddf2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:27.193441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab27ed-a917-4cd7-b4b4-a6b102eddf2e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:27.252720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 492/989] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:26:27 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:28.552539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566d020-918d-416d-89f5-77c407d65b27 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.553066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566d020-918d-416d-89f5-77c407d65b27 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.553066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566d020-918d-416d-89f5-77c407d65b27 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.554861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 498/990] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:28.560372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e6f33-7898-49b1-b689-54d920baa1eb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.560767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e6f33-7898-49b1-b689-54d920baa1eb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.560931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e6f33-7898-49b1-b689-54d920baa1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.580136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c70e6f33-7898-49b1-b689-54d920baa1eb None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:28.581369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c70e6f33-7898-49b1-b689-54d920baa1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:28.606334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:28.658447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 493/991] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:28.933167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566d020-918d-416d-89f5-77c407d65b27 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:28.945453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543433a4-8df5-4a92-b61e-6034ef1ffccb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.945762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543433a4-8df5-4a92-b61e-6034ef1ffccb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.946026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543433a4-8df5-4a92-b61e-6034ef1ffccb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.947710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 494/992] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:28.952801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:28.953079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:28.953345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:28.971432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:28.972634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:28.973626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.045219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566d020-918d-416d-89f5-77c407d65b27 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.060408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 495/993] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:29.081417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.082047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.082345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.082624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] PATH_INFO: `/v3/services/image` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.083203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.084710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d756af12-fe4d-4677-8917-7eac60c2a71a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:29.094085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:29.095045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:29.095922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.100169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 499/994] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Aug 24 16:26:28 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:29.101634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.172850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80cca0-970e-4f9e-bf7c-c31c1e545d8b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.173034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80cca0-970e-4f9e-bf7c-c31c1e545d8b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.173309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80cca0-970e-4f9e-bf7c-c31c1e545d8b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.174637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 500/995] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:29.179512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.179792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.180064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.193634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.198245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:29.199432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:29.210960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1854d-4a1c-4154-b367-d486598b4baa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.246823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.247423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.247698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.247965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.248616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] RBAC: Adding query filter params (name=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:29.248915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.250293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.255624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 496/996] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:29.261566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.272088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 501/997] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:29.334525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.362127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.370788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491681bf-f1df-46b5-8bef-5a1edaf6f185 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.406346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.406944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.407220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.407487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.408112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:29.408432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.409816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.414415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 497/998] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:29.455279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d488a0-987c-4cbd-beab-db892ac33130 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.494303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.495174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.495663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.496152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.497249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.499096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:29.509964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:29.511032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.512265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 502/999] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:29.517180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.534490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.553400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.628500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8148e2-c756-4f8b-8b69-49ac2cb70da5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.656525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.665724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341aa51-4a2d-4899-9afd-072227334c42 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.666749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.667527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.669231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.669231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.669513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:29.669749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.671260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.681066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 498/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:29.682279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.703348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.703984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.704291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.704779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.705442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.706585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:29.716460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:29.717527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:29.718396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 503/1001] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:29.723228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.769709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.794186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8c30-eb83-4701-b73e-1e02853940fc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.843913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0da8-4853-43dc-aec5-6cfaf6cc0003 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:29.847544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.848494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.848939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.849383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.850948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:29.851429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.853795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.868384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 499/1002] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:29.868384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.893609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:29.894226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:29.894505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:29.894774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:29.895409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:29.895710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:29.897246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:29.907628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 504/1003] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:29.908677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:29.932326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:29.985785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19605fb7-8e42-4d91-8181-5595079e8d41 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.016228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a61db8-9d1a-4f58-98a4-aba6ba1fcf6b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.023155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.023763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.024039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.024309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.024990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.026370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.052495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.053099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.053489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.054349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.054971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:30.055268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.056631 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.061595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 505/1004] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:30.067323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.099248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:30.104801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 500/1005] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:30.165174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:30.182680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3722d-e5c2-46e6-93e8-9f5677676c74 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.225877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.226484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.226771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.227027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.227727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] RBAC: Adding query filter params (interface=public, service_id=e1a7e56a3c654bf183cb581faf8ef131, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:30.228031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.229439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c4164-9615-4d4d-ae5a-fb20233586db None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.235296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 506/1006] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:30 2022] GET /identity/v3/endpoints?service_id=e1a7e56a3c654bf183cb581faf8ef131&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:30.240770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:30.353098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044aa51-2798-4b9a-935c-b9d44dac4cc4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:30.387632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6666f8-c03d-4336-b9a3-fa02014fcb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:30.388248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6666f8-c03d-4336-b9a3-fa02014fcb87 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:30.388622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6666f8-c03d-4336-b9a3-fa02014fcb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:30.388895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6666f8-c03d-4336-b9a3-fa02014fcb87 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:30.389510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6666f8-c03d-4336-b9a3-fa02014fcb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:30.390908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6666f8-c03d-4336-b9a3-fa02014fcb87 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:30.397723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 501/1007] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:30 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:26:31.366838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b89a0e-6414-4920-a4fb-f0dd874c4269 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.367386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b89a0e-6414-4920-a4fb-f0dd874c4269 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.367386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b89a0e-6414-4920-a4fb-f0dd874c4269 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.368908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 507/1008] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:31.373929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1c291-457a-4370-8386-0c40a11730ae None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.374312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1c291-457a-4370-8386-0c40a11730ae None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.374506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1c291-457a-4370-8386-0c40a11730ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.393087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fce1c291-457a-4370-8386-0c40a11730ae None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:31.394287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fce1c291-457a-4370-8386-0c40a11730ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:31.458582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:31.468118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 502/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:31.740427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b89a0e-6414-4920-a4fb-f0dd874c4269 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:31.844701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b89a0e-6414-4920-a4fb-f0dd874c4269 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:31.879022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3e375-5f36-4246-a64a-df8ddb47f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:31.879627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3e375-5f36-4246-a64a-df8ddb47f567 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:31.879906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3e375-5f36-4246-a64a-df8ddb47f567 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:31.880179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3e375-5f36-4246-a64a-df8ddb47f567 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:31.880855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3e375-5f36-4246-a64a-df8ddb47f567 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:31.882937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3e375-5f36-4246-a64a-df8ddb47f567 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:31.989315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 508/1010] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:26:31 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:32.048104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817b35-c9c1-45b5-81bb-ffcfe05c57c4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.048510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817b35-c9c1-45b5-81bb-ffcfe05c57c4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.048696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b817b35-c9c1-45b5-81bb-ffcfe05c57c4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.050262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 503/1011] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:32.055990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7fae0-35f9-461e-a27f-4cbfde95f319 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.056311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7fae0-35f9-461e-a27f-4cbfde95f319 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.056576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7fae0-35f9-461e-a27f-4cbfde95f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.064465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.078165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a3f7fae0-35f9-461e-a27f-4cbfde95f319 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:32.079392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a3f7fae0-35f9-461e-a27f-4cbfde95f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:32.166144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 509/1012] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:32 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:32.327310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a9c3-5133-4c9b-bce8-3b95469a6f57 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.327610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a9c3-5133-4c9b-bce8-3b95469a6f57 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.327774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a9c3-5133-4c9b-bce8-3b95469a6f57 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.329272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 504/1013] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:32.335532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.335838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.336186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.355174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:32.356373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:32.425081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 510/1014] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:32 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:32.437954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50a9c3-5133-4c9b-bce8-3b95469a6f57 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.544964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50a9c3-5133-4c9b-bce8-3b95469a6f57 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.580328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.580926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.581202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.581469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.582582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.584038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.585851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 505/1015] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:32.591374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.650819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.691569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491f52-abbe-47ff-845f-0afa8e082ab2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.702118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:32.731738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.732381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.732666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.732935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.733608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.734996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.771466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:32.811918 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d143d2-c69a-4311-b48a-d942be6c0841 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:32.844602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 511/1016] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:26:32 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:32.856276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:32.856988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:32.857332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:32.857658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:32.858318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:32.859828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:32.872607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:32.873707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 16:26:32.874554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:32.876733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 506/1017] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:32.880042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.003845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86ea23-3cfe-4945-b86f-e5630b56f1a5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.017407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.042064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.042672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.042950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.043295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.043896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:33.044194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.045620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.050963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 512/1018] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:33.056729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.167093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4e377-31e7-4026-b691-7c2909d19824 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.193099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:33.202636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.203242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.203520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.203787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.204450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:33.207222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.207222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634617d9-2148-433c-8cfa-3dc9b35f3394 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.211557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 507/1019] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Aug 24 16:26:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:33.217211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:33.321140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72773b1-8dea-4e4c-8281-aa81035996e7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:33.355375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb7a8d-7099-4cb9-942d-b772fb5eb38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:33.355972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb7a8d-7099-4cb9-942d-b772fb5eb38a None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:33.356246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb7a8d-7099-4cb9-942d-b772fb5eb38a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:33.356538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb7a8d-7099-4cb9-942d-b772fb5eb38a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:33.357193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb7a8d-7099-4cb9-942d-b772fb5eb38a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:33.358614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb7a8d-7099-4cb9-942d-b772fb5eb38a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:33.440458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 513/1020] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:34.005950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908034-60d5-42f8-a943-bbefb6fccd25 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.006662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908034-60d5-42f8-a943-bbefb6fccd25 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.007015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908034-60d5-42f8-a943-bbefb6fccd25 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.008434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 508/1021] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:34.013614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.013903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.014180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.033014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:34.034197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:34.060956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:34.108374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 514/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:34.387568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908034-60d5-42f8-a943-bbefb6fccd25 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.491902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908034-60d5-42f8-a943-bbefb6fccd25 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.526087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:34.526724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.526997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.527261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.528105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.529550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:34.538803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:34.539804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:34.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:34.542144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 509/1023] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:34.545856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.649359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efc46f-94b1-480d-a395-8bf9ede594dc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.685330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:34.686220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.686736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.687198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.688077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:34.688585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.690132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43148ba-3f1c-40b7-9d2b-3073f35062c6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.700108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 515/1024] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:34.701359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.762026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:34.806469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24af2b-1e0b-4ec6-85d2-2c25075a6763 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.832558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf8a40-2ad1-41f4-93ba-66ebe7714a95 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.832871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf8a40-2ad1-41f4-93ba-66ebe7714a95 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.833096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf8a40-2ad1-41f4-93ba-66ebe7714a95 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.834554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 516/1025] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:34.839647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.839928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.840197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.843730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:34.844334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:34.844684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:34.844891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:34.845518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:34.845814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:34.847184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:34.852564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 510/1026] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:34.858284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:34.858401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:34.859450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:34.876882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:34.937996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 517/1027] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:34.975188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b4a8-e4b9-49b6-bb75-94903239b7c0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:34.993565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:35.011095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:35.011707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:35.011963 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:35.012230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:35.012956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] RBAC: Adding query filter params (interface=public, service_id=c3c0b89792aa4dcd811a6d97fdb08bd5, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:35.013262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:35.014655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:35.024548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 511/1028] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:26:34 2022] GET /identity/v3/endpoints?service_id=c3c0b89792aa4dcd811a6d97fdb08bd5&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:35.025664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:35.128235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bae9cd-41c3-4bec-a0b5-8b2d73ad8155 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:35.163007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:35.163645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:35.163922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:35.164188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:35.164828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:35.166178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:35.180641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 518/1029] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:26:35 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:26:35.222878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:35.331492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd4a2e-196f-4a0f-b4ea-71da8373d6c8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:35.368182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14602dbe-a759-4348-8214-49e88cc942cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:35.369182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14602dbe-a759-4348-8214-49e88cc942cb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:35.369615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14602dbe-a759-4348-8214-49e88cc942cb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:35.369884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14602dbe-a759-4348-8214-49e88cc942cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:35.371115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14602dbe-a759-4348-8214-49e88cc942cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:35.372690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14602dbe-a759-4348-8214-49e88cc942cb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:35.374325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 512/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:35.380189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:35.404270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:35.486842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f0468-5124-4412-aaed-ba54b9c5f86f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:35.505927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51847949-eb1d-4217-95e2-bfa02d70010a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:35.506263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51847949-eb1d-4217-95e2-bfa02d70010a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:35.506501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51847949-eb1d-4217-95e2-bfa02d70010a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:35.508046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 513/1031] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:35.513322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:35.513606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:35.513881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:35.525947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:35.526565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:35.526848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:35.527126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:35.528046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:35.529527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:35.533622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:35.535029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:35.596362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:35.617612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 514/1032] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:35.624378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 519/1033] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:35 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:35.899423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.000793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123b4ee-f3b0-4aaa-b04d-3de6e3b5bc9e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:36.035596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:36.036242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.036565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.036844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.046984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:36.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:36.052821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:36.053937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 515/1034] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Aug 24 16:26:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:36.060672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.133356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:36.164129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c39d2-0740-4469-9ba1-6287a8fffed3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:36.199008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:36.201109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.201109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.201109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.201109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:36.201109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:36.202464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:36.215644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 520/1035] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:26:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:36.216950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.261272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:36.322957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c29eb9-e5b6-482b-9dce-d98e3f7d5c8b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:36.358642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:36.359266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.359539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.359811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.369751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:36.371215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:36.385667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:36.386593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:36.392333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:36.392333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:36.392333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:36.392333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 516/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:36.392924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.491908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3248db7-7f20-4bd9-a1b2-e78600e3d695 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:36.526271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:36.526931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.527210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.527475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.528400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:36.528700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:36.530049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:36.531575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:36.547436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 521/1037] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:36.548703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.641342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:36.651636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15dbc3-f9c1-4a1f-8ac8-41ced76d72f8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:36.685588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:36.686219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.686498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.686753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.694925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:36.696406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:36.707373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:36.708654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:36.710534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:36.711557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 517/1038] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:36.715080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.828187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751aec-7ecb-4837-baac-f70a94e6b59d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:36.863228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:36.863841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:36.864118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:36.864412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:36.865143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:36.865474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:36.866897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9539aaa-c145-4638-81e9-17ae6277cb55 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:36.886834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 522/1039] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:36.892410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:36.999876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a32c90-8d46-434b-8dd1-a162a3db4e97 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.035914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.036598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.036821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.037162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.059879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.106955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.108366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.177553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6616efc-66b2-47ab-a045-4fbb36800606 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.177995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6616efc-66b2-47ab-a045-4fbb36800606 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.178415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6616efc-66b2-47ab-a045-4fbb36800606 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.179891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 523/1040] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:26:37.184980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.185263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.185537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.203104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:37.204271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:37.213323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 518/1041] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:37.218906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.259400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.278469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.282438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 524/1042] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:26:37.327593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a20fba-8165-4f5a-9caa-930ffc986280 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.362304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.362839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.363114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.363379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.373524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.374986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:37.389991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:37.390941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:37.391900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:37.391900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:37.391900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:37.392637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 519/1043] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:37.397376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.498418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da628460-078d-47e1-a184-73bf81b6d7b4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.533463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.534063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.534339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.534605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.535245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:37.535544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.536936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.553289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 525/1044] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:37.555136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.556429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.605868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.662222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76248-93e1-43aa-b402-832a32f13bf1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.664833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.671224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bead8-b84f-49e8-bc3c-bb1de886f232 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.697221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.697842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.698115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.698380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.706994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.707181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.707663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.707940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.708209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] PATH_INFO: `/v3/services/image` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.708903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.709384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.710410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e277eba1-ee65-4ab6-b448-93345fef135c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:37.719370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:37.721379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:37.722542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 526/1045] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:37.723515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3590ac-c311-4c35-94ef-ff4d4b504ef8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.723515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3590ac-c311-4c35-94ef-ff4d4b504ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.723515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3590ac-c311-4c35-94ef-ff4d4b504ef8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.725578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 527/1046] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d4725be2-2348-4eb6-8f05-5b7c8a069ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:37.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:37.726980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:37.727860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 520/1047] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:37.732400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3590ac-c311-4c35-94ef-ff4d4b504ef8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.733265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.733540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.733807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.752185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:37.753413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:37.821128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 521/1048] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:37.822012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.846218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3590ac-c311-4c35-94ef-ff4d4b504ef8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.859043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.900727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.902205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.902497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.903029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.903675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] RBAC: Adding query filter params (name=image) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:37.903975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.905440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:37.923499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 528/1049] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:37.923499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:37.936421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:37.946587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0c2a-fe7f-4a5e-96c2-60fc5567d813 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:37.992575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:37.993516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:37.993846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:37.994034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:37.994732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:37.995030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:37.996499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.008566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 522/1050] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.016621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.057021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96418eba-ce61-48a0-8250-b417a5b2ecfd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.097700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.098395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.098676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.098940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.099569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] RBAC: Adding query filter params (type=image) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:38.099864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.101461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.106697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 529/1051] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.114064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.150300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0848d33-bf82-4798-a963-c1a43830497e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.151033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.191085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.191602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.191873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.192135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.192895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] RBAC: Adding query filter params (role.id=adc0f1e2c62f4960b6cff713d64e608f, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=4542cf167cc24738b441af6530dead3c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:38.193721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.195536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.201533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 523/1052] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/role_assignments?user.id=4542cf167cc24738b441af6530dead3c&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=adc0f1e2c62f4960b6cff713d64e608f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.212657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.235366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f13db9-4c19-4918-9a92-ddaa4da6bb26 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.242294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.296878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.297519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.297799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.298071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.298796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.305370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.361235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61739f-1bbe-4c5a-a4c9-0b59b16b48a2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.400782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.401421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.401706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.401975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.414531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:38.420082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:38.421214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:38.422167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:38.423854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 524/1053] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:38.430424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.434956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 530/1054] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:26:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:26:38.529108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.556950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fee030-17cb-463b-9c11-703579010a45 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.616314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.617047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.617608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.617964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.618722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:38.619089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.620565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.630697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 525/1055] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:38.638950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.768455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:38.783881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eae56-f344-4c90-b46c-9ba03e288990 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.821128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.822353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.822353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.822643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] PATH_INFO: `/v3/users/7372909b6ca541b2804cf74431196f5a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:38.837327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] RBAC: Authorizing `identity:get_user(user_id=7372909b6ca541b2804cf74431196f5a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:38.839651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:38.841729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 531/1056] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/users/7372909b6ca541b2804cf74431196f5a => generated 285 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:38.847642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:38.958513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f0fe0-628c-4c1c-b420-e4ba9ae87f66 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:38.995297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:38.995906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:38.996183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:38.996474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:39.000816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:39.068117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:39.069571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:39.172678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 526/1057] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:39.179403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:39.279625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016e412-e93a-4ab1-9402-fffcf0bc6634 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:39.313853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038ae85-b794-4006-9721-a1dc20c2c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:39.314472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ae85-b794-4006-9721-a1dc20c2c37f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:39.314748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ae85-b794-4006-9721-a1dc20c2c37f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:39.315012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ae85-b794-4006-9721-a1dc20c2c37f None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:39.316263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038ae85-b794-4006-9721-a1dc20c2c37f None admin] RBAC: Authorizing `identity:get_project(project_id=29bd9b62b48b412288a03ee3a7b69e9d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:39.317704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038ae85-b794-4006-9721-a1dc20c2c37f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:39.319839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 532/1058] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:39 2022] GET /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:39.325608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:39.361172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:39.431886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee7664-7e01-4048-8f02-6624de659286 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:39.466999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:39.467739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:39.468113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:39.468512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:39.469336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d, user.id=7372909b6ca541b2804cf74431196f5a) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:39.469742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:39.471288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189eec2d-21e1-4ec6-a16b-0d5452171bfa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:39.477641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 527/1059] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:39 2022] GET /identity/v3/role_assignments?user.id=7372909b6ca541b2804cf74431196f5a&scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:40.219376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601199-817f-4f91-8465-0ee9078ccd2e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.219865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601199-817f-4f91-8465-0ee9078ccd2e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.219865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2601199-817f-4f91-8465-0ee9078ccd2e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.221357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 533/1060] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:40.226458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686468-490c-4815-8e38-780a8611fd48 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.226888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686468-490c-4815-8e38-780a8611fd48 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.226977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4686468-490c-4815-8e38-780a8611fd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.245369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f4686468-490c-4815-8e38-780a8611fd48 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:40.246538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4686468-490c-4815-8e38-780a8611fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:40.319381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 528/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:40.444372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a6b22-4419-43ba-9311-e9811fb6df63 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.444724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a6b22-4419-43ba-9311-e9811fb6df63 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.444917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a6b22-4419-43ba-9311-e9811fb6df63 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.446399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 534/1062] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:40.451443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.451729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.451999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.470218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:40.471320 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:40.497130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:40.543877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 529/1063] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:40.583120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a6b22-4419-43ba-9311-e9811fb6df63 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:40.602411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:40.688798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a6b22-4419-43ba-9311-e9811fb6df63 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:40.723961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:40.725458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.725904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.726504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.737151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:40.742313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:40.743441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:40.744659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:40.746920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 535/1064] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Aug 24 16:26:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:40.752490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:40.819989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:40.861856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e7fc8-7d81-463c-9e75-1f752f4857c0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:40.904262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:40.904976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.905248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.905514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:40.906276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:40.906583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:40.908089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:40.909776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:40.920489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 530/1065] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:26:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:40.924718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:40.934012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660b9a3-bb8e-4346-bb71-a40ce9df66f3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:40.951535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:40.970960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:40.971565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:40.972000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:40.972269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.032919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ece887-3e56-40fa-bf85-058910194906 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.046910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.048628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.069731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.070360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.070633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.070897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.082253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.084301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:41.099146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.100202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:41.101188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:41.101188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.101188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:41.101188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 531/1066] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:41.105848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.133949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.158140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 536/1067] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:41.163882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.189568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.217262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd88b96-4c2e-47f7-8d99-c575acd2cff5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.254687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.255349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.256081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.256363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.257018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:41.257314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.258925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.264718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44482026-46f2-452a-9abc-5533ea0ee92d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.273078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 532/1068] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:41.278832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.304729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.305670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.306098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.306521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.317124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.319301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:41.335710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.337136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:41.338251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:41.338251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.338251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:41.340032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 537/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:41.345727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.378187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.395848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d97572-33e3-4cd4-96ba-66b837d86a29 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.433566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.434194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.434476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.434748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.436716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.442202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.445534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0860400c-ab9a-4040-bccc-e232e74f2b15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:41.456255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.457250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:41.458434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:41.459627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba347f8b-e3f4-42ca-a4fd-11a30a89e82e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.459704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 533/1070] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:41.459704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81320ea3-7138-49c9-833f-078fc61907ed None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.459826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81320ea3-7138-49c9-833f-078fc61907ed None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.460145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81320ea3-7138-49c9-833f-078fc61907ed None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.461571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 534/1071] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:41.465207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81320ea3-7138-49c9-833f-078fc61907ed None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.495081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.495688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.495964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.496234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.497167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:41.497279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.498545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10a24-5bbf-4bea-b8b3-4040dfeb038c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.510087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 538/1072] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:41.511275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.511560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.511830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.531672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:41.533209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:41.578421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81320ea3-7138-49c9-833f-078fc61907ed None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.610170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 539/1073] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:41.627995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.628648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.628922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.629188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.629931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:41.630231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.632188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.661300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 535/1074] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:41.669482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.694347 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.783452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c1133-3d95-4d26-bd8b-5233b5dbf1e3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.822882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.823508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:41.823780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:41.824043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:41.835231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:41.836750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:41.851602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:41.852742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:41.853633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:41.853633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:41.853633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:41.853633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 540/1075] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:41.858461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.890802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:41.915946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.963194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:41.964998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935cf44-a4ea-402e-8ec4-7b6e155450d4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.993269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9843a-a0cd-4721-8ebf-e170df40ff32 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:41.999162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:41.999766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.000037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.000646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.000972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.001268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.002686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.016284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 536/1076] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:42.022474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.034736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.035251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.035528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.035806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.047012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:42.052646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.053821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:42.054828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 541/1077] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:42.061999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.131368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd57a0-b5cc-46ed-b6b3-fca5ac5bd695 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.144323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.165265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.165872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.166145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.166411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.172728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.174176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:42.186751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:42.187833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:42.188956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 537/1078] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:42.194664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.216525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7fe78-2501-43fc-b9f5-176ac1a9066e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.245050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.250785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.251386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.251667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.251933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.252636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.252933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.254292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.264853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 542/1079] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:42.270624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.292989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241546-7ded-4197-b238-c973b15fba82 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.326941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.327549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.327821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.328085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.328815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:42.329123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.330564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.342410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 538/1080] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:42.348472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.375702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3baefe-7f15-47dd-9fbd-70eb697f954c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.388396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.421175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.422102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.422518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.422929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] PATH_INFO: `/v3/users/7372909b6ca541b2804cf74431196f5a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.424649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] RBAC: Authorizing `identity:get_user(user_id=7372909b6ca541b2804cf74431196f5a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.426915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.432409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 543/1081] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/users/7372909b6ca541b2804cf74431196f5a => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:42.437222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.461843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f825f-4578-4716-b052-7aade14291bc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.497407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.498061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.498329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.498595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/4542cf167cc24738b441af6530dead3c/roles/adc0f1e2c62f4960b6cff713d64e608f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.500770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=4542cf167cc24738b441af6530dead3c, role_id=adc0f1e2c62f4960b6cff713d64e608f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.500889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.502220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.542559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98486bdd-cb29-4d01-b0f7-bd24765968f9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.563261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 539/1082] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:42 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/4542cf167cc24738b441af6530dead3c/roles/adc0f1e2c62f4960b6cff713d64e608f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:26:42.576494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a376c-1016-46d4-bcaf-dcca32444149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.577151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a376c-1016-46d4-bcaf-dcca32444149 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.577430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a376c-1016-46d4-bcaf-dcca32444149 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.577702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a376c-1016-46d4-bcaf-dcca32444149 None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.578899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a376c-1016-46d4-bcaf-dcca32444149 None admin] RBAC: Authorizing `identity:get_project(project_id=29bd9b62b48b412288a03ee3a7b69e9d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.580389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a376c-1016-46d4-bcaf-dcca32444149 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.582220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 544/1083] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:42 2022] GET /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:42.588663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:42.687682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:42.696520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f21779-5de2-4ec7-a8b2-a39ab650b9fd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:42.733848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bef302-12cb-4377-8ce1-1e4e29ea6dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:42.734604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bef302-12cb-4377-8ce1-1e4e29ea6dfa None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:42.734879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bef302-12cb-4377-8ce1-1e4e29ea6dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:42.735190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bef302-12cb-4377-8ce1-1e4e29ea6dfa None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d/users/7372909b6ca541b2804cf74431196f5a/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:42.737147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bef302-12cb-4377-8ce1-1e4e29ea6dfa None admin] RBAC: Authorizing `identity:create_grant(project_id=29bd9b62b48b412288a03ee3a7b69e9d, user_id=7372909b6ca541b2804cf74431196f5a, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:42.738615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bef302-12cb-4377-8ce1-1e4e29ea6dfa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:42.845837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 540/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:42 2022] PUT /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d/users/7372909b6ca541b2804cf74431196f5a/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:44.448438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a432c-06aa-4170-8393-d06034d435cb None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.448971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a432c-06aa-4170-8393-d06034d435cb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.448971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a432c-06aa-4170-8393-d06034d435cb None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.450489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 545/1085] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:44.455397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835cebd-171c-4645-aeb6-e7a57500dcc8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.455873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835cebd-171c-4645-aeb6-e7a57500dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.456078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835cebd-171c-4645-aeb6-e7a57500dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.475135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f835cebd-171c-4645-aeb6-e7a57500dcc8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:44.476331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f835cebd-171c-4645-aeb6-e7a57500dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:44.544518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 541/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:44.728609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83155d2-6623-4161-8c9a-d761b3a61772 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.728808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83155d2-6623-4161-8c9a-d761b3a61772 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.729067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83155d2-6623-4161-8c9a-d761b3a61772 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.730397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 546/1087] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:44.735711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.735997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.736268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.754875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:44.756681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:44.822954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 542/1088] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:44.823353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83155d2-6623-4161-8c9a-d761b3a61772 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:44.927878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83155d2-6623-4161-8c9a-d761b3a61772 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:44.954207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:44.962211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:44.962835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:44.970551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:44.970551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:44.973196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:44.978760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:44.979745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:44.980620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 547/1089] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:44.985929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.000855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.085026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2b3a0-dbce-4ed0-a0c5-c160045e1739 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.096450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.120555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.121165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.121439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.121711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.122391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.122690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.124144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.132399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 543/1090] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:26:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:45.137958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.287784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0b2ac-58b0-4d8c-9ba1-87d3c96cd17b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.294602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.318779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eecb54-1c90-4a87-8847-1ecbe90e5f95 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.321034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.321661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.321938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.322207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.332165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:26:45.337587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.338693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:26:45.339555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 548/1091] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:45.344638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.355266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.355890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.356162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.356451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.367109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.368607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:45.382386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.383532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:45.384670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:45.384670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:45.384670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:45.385052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 544/1092] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:45.389293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.394944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.445784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f50475-2b5d-48be-99ae-7e05975068e8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.480557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.481180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.481418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.481688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.482339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.482636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.484043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.492012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 549/1093] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:45.495288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58aca60-71b7-47bc-be99-d7a62d983bcf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.497774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.520443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.530149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.530778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.531048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.531313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.531947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.532247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.533663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.550506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 545/1094] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:45.551540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.597982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80b674-311a-4231-8c34-6c1e073b7749 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.608362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.631674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.632309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.632615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.632886 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] PATH_INFO: `/v3/users/7372909b6ca541b2804cf74431196f5a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.634016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] RBAC: Authorizing `identity:get_user(user_id=7372909b6ca541b2804cf74431196f5a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.635429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.637030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 550/1095] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/users/7372909b6ca541b2804cf74431196f5a => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:45.642864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.655319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421d804-0d1a-4f1c-b508-ca81888e40cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.696883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.697522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.697793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.698059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.705244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.706702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:26:45.718487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:45.719740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:45.720908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 546/1096] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:45.725498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.746097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682e52c-ee71-4297-a7dd-276374ac9f39 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.771994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.780504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.781101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.781379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.781648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.826831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84a644-e616-4037-bf68-453344b41850 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.827956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:45.849987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.851432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.862046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:45.862649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:45.862923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:45.863187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:45.863916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:45.864216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:45.866525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:45.883777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 547/1097] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:45.889920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.955640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 551/1098] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:26:45.962630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:45.991548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a637a-afa5-4a25-a24d-0fb86bc23a64 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:45.994623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.026799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.027656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.027931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.028199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.040758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.061505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c69fb-61ff-4e2a-8c46-78a41ff2b9a3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.095970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.096531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.096660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.096826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.097096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.097999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.098210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] RBAC: Authorizing `identity:get_project(project_id=29bd9b62b48b412288a03ee3a7b69e9d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.100168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.101896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 552/1099] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:46.107915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.198528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 548/1100] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 16:26:46.204186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.206797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.208917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f07868-acbb-4fe7-969a-482e0634e784 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.242863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.243453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.243727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.243991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.244736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d, user.id=7372909b6ca541b2804cf74431196f5a) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.245033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.246445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.252040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 553/1101] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/role_assignments?user.id=7372909b6ca541b2804cf74431196f5a&scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.305719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba1493-7908-458d-9dd8-0fd1a5780d22 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.342872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.343512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.343785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.344052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.356010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.357450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:26:46.373447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:46.374666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:26:46.375794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:26:46.375794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:26:46.375794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:46.376683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 549/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:26:46.380803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.394186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.487891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b18415-4c01-424d-acc1-cf0af74cea6e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.521349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.521954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.522234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.522507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.523148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.523449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.524860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.537656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 554/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.543120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.598264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.642652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d6de-0a8b-4afc-9d8b-2abbc37c07c3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.675814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.676461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.676741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.677565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.684314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.685812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:26:46.697905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:26:46.698977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:26:46.699910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 550/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:26:46.704851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.779470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:46.816950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8412a-d30a-497b-bce4-c6ae2bfb2cdc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.850909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:46.851505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:46.851786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:46.852058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:46.852777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:46.853084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:46.854495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51145c7d-b04f-45ab-8b69-77ca2bb14378 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:46.866038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 555/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:46.872097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:46.980092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14374566-5a17-4447-abed-65afc7c4e66e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:46.995604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:47.013328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:47.013952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:47.014227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:47.014492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:47.015216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] RBAC: Adding query filter params (role.id=adc0f1e2c62f4960b6cff713d64e608f, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=4542cf167cc24738b441af6530dead3c) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:47.015520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:47.017010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ed0fb-a288-43d3-9abe-be96207a325b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:47.023328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 551/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:46 2022] GET /identity/v3/role_assignments?user.id=4542cf167cc24738b441af6530dead3c&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=adc0f1e2c62f4960b6cff713d64e608f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:48.198338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd5b8-d84e-4e46-8dd8-aeec2cbb550d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.198850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd5b8-d84e-4e46-8dd8-aeec2cbb550d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.198850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd5b8-d84e-4e46-8dd8-aeec2cbb550d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.200136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 556/1107] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:48.205437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.205855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.205983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.224699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:48.225875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:48.290021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 552/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:48.559834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd5b8-d84e-4e46-8dd8-aeec2cbb550d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.659634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd5b8-d84e-4e46-8dd8-aeec2cbb550d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.686509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:48.691879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b316e-c886-4cdf-be80-4325b7f9a2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.693040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b316e-c886-4cdf-be80-4325b7f9a2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.693313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b316e-c886-4cdf-be80-4325b7f9a2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.693575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b316e-c886-4cdf-be80-4325b7f9a2f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.694681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b316e-c886-4cdf-be80-4325b7f9a2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.696120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b316e-c886-4cdf-be80-4325b7f9a2f1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.697679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 557/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:26:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:48.703328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:48.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62085d07-5b86-4c66-9d36-9d39d3cc3cb4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:48.811772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:48.835042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552293dc-e42a-47bc-a168-384f8ebb757f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:48.835684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552293dc-e42a-47bc-a168-384f8ebb757f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:48.835915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552293dc-e42a-47bc-a168-384f8ebb757f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:48.836179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552293dc-e42a-47bc-a168-384f8ebb757f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:48.836848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552293dc-e42a-47bc-a168-384f8ebb757f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:48.838275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552293dc-e42a-47bc-a168-384f8ebb757f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:48.952950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 553/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:26:48 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:26:50.891924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70087364-28b0-4315-9ed3-7fe23ed19749 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.892460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70087364-28b0-4315-9ed3-7fe23ed19749 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.892551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70087364-28b0-4315-9ed3-7fe23ed19749 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.893919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 558/1111] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:50.900969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:50.901595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:50.901595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:50.919849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:50.921063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:50.987723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 554/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:51.272012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70087364-28b0-4315-9ed3-7fe23ed19749 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.370723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70087364-28b0-4315-9ed3-7fe23ed19749 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.403388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.403995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.404271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.404585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] PATH_INFO: `/v3/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.405965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] RBAC: Authorizing `identity:get_role(role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.407325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.408657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 559/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/roles/df3ae0f0715049bf9ef8923719615f92 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.414561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.506420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.512072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81489e-d4b2-45ee-8619-058d75048f42 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.545732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.546293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.546566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.546876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] PATH_INFO: `/v3/users/9952f5a42fe54d50945e6615ce732242` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.560109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] RBAC: Authorizing `identity:get_user(user_id=9952f5a42fe54d50945e6615ce732242)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.561588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.563250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 555/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/users/9952f5a42fe54d50945e6615ce732242 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:51.569014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.632801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.667790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f6233-2919-4c32-8c76-bd8f353ac243 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:51.700072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:51.700688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:51.700964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:51.701232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:51.779711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:51.781184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:51.843008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:51.877087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 560/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:51.883295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:51.983239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187afeb-ddbc-4932-85c0-b85830d3adcd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.016579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63d56a-0e4c-4419-8724-3cf4e9972f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.017193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63d56a-0e4c-4419-8724-3cf4e9972f6e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.017467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63d56a-0e4c-4419-8724-3cf4e9972f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.017736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63d56a-0e4c-4419-8724-3cf4e9972f6e None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.018831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63d56a-0e4c-4419-8724-3cf4e9972f6e None admin] RBAC: Authorizing `identity:get_project(project_id=29bd9b62b48b412288a03ee3a7b69e9d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.020302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63d56a-0e4c-4419-8724-3cf4e9972f6e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.021935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 556/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:51 2022] GET /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:52.028331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:52.124127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9062d-1fe4-407c-a82d-a7260d2a87e2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:52.156882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:52.157470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:52.157745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:52.158013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:52.158725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] RBAC: Adding query filter params (role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d, user.id=9952f5a42fe54d50945e6615ce732242) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:52.159027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:52.160453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8ff90-495a-4e4d-8d9e-9367d7632348 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:52.165676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 561/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:52 2022] GET /identity/v3/role_assignments?user.id=9952f5a42fe54d50945e6615ce732242&scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:26:54.035228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7aaa8-8263-4836-9160-bca9b8bd86e4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.036281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7aaa8-8263-4836-9160-bca9b8bd86e4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.036281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7aaa8-8263-4836-9160-bca9b8bd86e4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.037675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 557/1118] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:54.042847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.043274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.043375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.061472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:54.062649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:54.085327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.131337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 562/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:54.401251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7aaa8-8263-4836-9160-bca9b8bd86e4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.425217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.502780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7aaa8-8263-4836-9160-bca9b8bd86e4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.536650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.537270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.537542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.537806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] PATH_INFO: `/v3/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.539214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] RBAC: Authorizing `identity:get_role(role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.540614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.541905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 558/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/roles/df3ae0f0715049bf9ef8923719615f92 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:54.547698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.643348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784c20e-7bb1-4b90-9d1e-72bb0cc9107d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.676201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.677497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.677771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.678033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] PATH_INFO: `/v3/users/9952f5a42fe54d50945e6615ce732242` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.679135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] RBAC: Authorizing `identity:get_user(user_id=9952f5a42fe54d50945e6615ce732242)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.680565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.682103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 563/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/users/9952f5a42fe54d50945e6615ce732242 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:54.687745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.774344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.783548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462f9f6-9fc5-4c59-a31e-9e4b7612e81f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.817377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50fc18-e460-4116-997b-720750fd7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.817990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50fc18-e460-4116-997b-720750fd7481 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.818261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50fc18-e460-4116-997b-720750fd7481 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.818525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50fc18-e460-4116-997b-720750fd7481 None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.819619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50fc18-e460-4116-997b-720750fd7481 None admin] RBAC: Authorizing `identity:get_project(project_id=29bd9b62b48b412288a03ee3a7b69e9d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.821078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50fc18-e460-4116-997b-720750fd7481 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:54.822678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 559/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:54 2022] GET /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:54.828546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:54.856377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:54.926093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e22658-1d11-4a3c-a3ad-04ae750b1f9b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:54.960083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da6ac3-e337-41d3-9843-0a646983ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:54.960689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da6ac3-e337-41d3-9843-0a646983ea41 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:54.960962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da6ac3-e337-41d3-9843-0a646983ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:54.961227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da6ac3-e337-41d3-9843-0a646983ea41 None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d/users/9952f5a42fe54d50945e6615ce732242/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:54.963283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da6ac3-e337-41d3-9843-0a646983ea41 None admin] RBAC: Authorizing `identity:create_grant(project_id=29bd9b62b48b412288a03ee3a7b69e9d, user_id=9952f5a42fe54d50945e6615ce732242, role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:54.964840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da6ac3-e337-41d3-9843-0a646983ea41 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:55.027726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 564/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:26:54 2022] PUT /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d/users/9952f5a42fe54d50945e6615ce732242/roles/df3ae0f0715049bf9ef8923719615f92 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:26:56.879961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27bb5c-bc13-406d-8df7-5f69b4270d04 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.880552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27bb5c-bc13-406d-8df7-5f69b4270d04 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.881203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27bb5c-bc13-406d-8df7-5f69b4270d04 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.882929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 560/1124] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:26:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:26:56.888222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:56.888693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:56.888805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:56.907201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:26:56.908388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:26:56.975531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 565/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:26:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:26:57.251932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27bb5c-bc13-406d-8df7-5f69b4270d04 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.374256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27bb5c-bc13-406d-8df7-5f69b4270d04 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.407672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.408291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.408603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.408875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] PATH_INFO: `/v3/roles/df3ae0f0715049bf9ef8923719615f92` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.410268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] RBAC: Authorizing `identity:get_role(role_id=df3ae0f0715049bf9ef8923719615f92)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.411946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.413293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 561/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/roles/df3ae0f0715049bf9ef8923719615f92 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.419080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.437213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.515839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a81854-dc34-4dfb-9eb0-ca9d967a21b3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.548787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.549390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.549663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.549928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] PATH_INFO: `/v3/users/9952f5a42fe54d50945e6615ce732242` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.551101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] RBAC: Authorizing `identity:get_user(user_id=9952f5a42fe54d50945e6615ce732242)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.553061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.554602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 566/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/users/9952f5a42fe54d50945e6615ce732242 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:57.560312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.586929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.663844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113d777-2803-4b19-8eb8-e3a7cdc03150 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:57.701090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:57.701688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:57.701959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:57.702226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:57.767376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:57.768789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:57.794570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.866747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 562/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:26:57.872682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:57.961471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:57.978396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b14a45-2884-41f3-b5ba-3a83a12e7940 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.017920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b9d90-dd25-4e24-9a9c-0a55d863a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.018462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b9d90-dd25-4e24-9a9c-0a55d863a3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.018735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b9d90-dd25-4e24-9a9c-0a55d863a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.018998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b9d90-dd25-4e24-9a9c-0a55d863a3d3 None admin] PATH_INFO: `/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.020377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b9d90-dd25-4e24-9a9c-0a55d863a3d3 None admin] RBAC: Authorizing `identity:get_project(project_id=29bd9b62b48b412288a03ee3a7b69e9d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.021788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b9d90-dd25-4e24-9a9c-0a55d863a3d3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.023398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 567/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:26:57 2022] GET /identity/v3/projects/29bd9b62b48b412288a03ee3a7b69e9d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:26:58.029836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:26:58.126178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6625a43-63dc-475a-a549-d1663bb69756 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:26:58.151376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:26:58.159835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:26:58.165063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:26:58.165063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:26:58.165063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:26:58.165063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] RBAC: Adding query filter params (role.id=df3ae0f0715049bf9ef8923719615f92, scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d, user.id=9952f5a42fe54d50945e6615ce732242) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:26:58.165063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:26:58.165063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2f4dd-2100-48c1-9aeb-b3d78a2d85fc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:26:58.168932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 563/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:26:58 2022] GET /identity/v3/role_assignments?user.id=9952f5a42fe54d50945e6615ce732242&scope.project.id=29bd9b62b48b412288a03ee3a7b69e9d&role.id=df3ae0f0715049bf9ef8923719615f92 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.060550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a2f8a-6983-4f6f-8bc3-f5748b20a8c1 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.061098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a2f8a-6983-4f6f-8bc3-f5748b20a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.061098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a2f8a-6983-4f6f-8bc3-f5748b20a8c1 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.062478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 568/1131] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:00.068472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.068911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.069117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.087414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:00.088618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:00.153351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 564/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:00.417647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a2f8a-6983-4f6f-8bc3-f5748b20a8c1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.516135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a2f8a-6983-4f6f-8bc3-f5748b20a8c1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.549622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de79c34-657a-4bce-a9b2-290447ae4558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.550227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de79c34-657a-4bce-a9b2-290447ae4558 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.550500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de79c34-657a-4bce-a9b2-290447ae4558 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.550854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de79c34-657a-4bce-a9b2-290447ae4558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.552132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de79c34-657a-4bce-a9b2-290447ae4558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.553585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de79c34-657a-4bce-a9b2-290447ae4558 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.555113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 569/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:00.560943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:00.655925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb39794-a375-47f5-af8d-d8848af2264b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:00.674999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:00.688200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac8a9d-fc02-4558-a6d1-0cd2bef327b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:00.688833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac8a9d-fc02-4558-a6d1-0cd2bef327b4 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:00.690855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac8a9d-fc02-4558-a6d1-0cd2bef327b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:00.690855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac8a9d-fc02-4558-a6d1-0cd2bef327b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:00.690855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac8a9d-fc02-4558-a6d1-0cd2bef327b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:00.692193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac8a9d-fc02-4558-a6d1-0cd2bef327b4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:00.791893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 565/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:00 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:02.700534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebf02f-3496-4d3b-a791-4752a9f4ec32 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.701091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebf02f-3496-4d3b-a791-4752a9f4ec32 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.701091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebf02f-3496-4d3b-a791-4752a9f4ec32 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.702392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 570/1135] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:02.708414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14677167-f711-4ba9-86d8-84473187a849 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:02.708827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14677167-f711-4ba9-86d8-84473187a849 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:02.708952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14677167-f711-4ba9-86d8-84473187a849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:02.726946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-14677167-f711-4ba9-86d8-84473187a849 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:02.728113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-14677167-f711-4ba9-86d8-84473187a849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:02.792883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 566/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:03.067092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebf02f-3496-4d3b-a791-4752a9f4ec32 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.096561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.171958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebf02f-3496-4d3b-a791-4752a9f4ec32 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.205602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bb15e-bb89-4bc7-bc2f-0052852e672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.206216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb15e-bb89-4bc7-bc2f-0052852e672b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.206494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb15e-bb89-4bc7-bc2f-0052852e672b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.206777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb15e-bb89-4bc7-bc2f-0052852e672b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.208022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bb15e-bb89-4bc7-bc2f-0052852e672b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.209478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bb15e-bb89-4bc7-bc2f-0052852e672b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.211038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 571/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:27:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:03.216802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14677167-f711-4ba9-86d8-84473187a849 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:03.311476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14677167-f711-4ba9-86d8-84473187a849 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:03.312970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:03.344807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8131f-685f-4119-a658-f63560453e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:03.345324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8131f-685f-4119-a658-f63560453e5a None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:03.345596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8131f-685f-4119-a658-f63560453e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:03.345860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8131f-685f-4119-a658-f63560453e5a None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:03.346493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8131f-685f-4119-a658-f63560453e5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:03.348079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8131f-685f-4119-a658-f63560453e5a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:03.432214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 567/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:03 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:05.345305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0016e-f597-413a-87c7-00d8b715d970 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.345866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0016e-f597-413a-87c7-00d8b715d970 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.345866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0016e-f597-413a-87c7-00d8b715d970 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.347742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 572/1139] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:05.352999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.353421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.353523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.372461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:05.373634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:05.437833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 568/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:05.706244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0016e-f597-413a-87c7-00d8b715d970 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:05.795425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:05.805873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0016e-f597-413a-87c7-00d8b715d970 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:05.838177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:05.838795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.839070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.839338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.850211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:05.854824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:05.856012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:27:05.857166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 573/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:05.861957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:05.950125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:05.962235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b488b4-32be-4253-9575-b66504e905e7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:05.994590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:05.995353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:05.995628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:05.995894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:05.996577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:05.996874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:05.998249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.006962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 569/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:06.012519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.108825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e1687-9fdc-4c3c-9e5d-361ac474da18 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.142105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.142821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.143098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.143364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] PATH_INFO: `/v3/users/ab9a96daebf94f2780001fe4b0e74fd4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.157360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] RBAC: Authorizing `identity:get_user(user_id=ab9a96daebf94f2780001fe4b0e74fd4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.164006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.164006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 574/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/users/ab9a96daebf94f2780001fe4b0e74fd4 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:06.166523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.265234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dfb8f-4a9b-4da8-9cdb-8ecfd087f7a9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.297570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.298158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.298428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.298691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.317709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.367302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.368810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.464902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 570/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:06.470836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.482174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.569846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e15bdc-9004-4054-8583-25773fb838af None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.602639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31982a-1fb1-4ed8-bd88-aa741a9c82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.603251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31982a-1fb1-4ed8-bd88-aa741a9c82ec None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.603526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31982a-1fb1-4ed8-bd88-aa741a9c82ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.603801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31982a-1fb1-4ed8-bd88-aa741a9c82ec None admin] PATH_INFO: `/v3/projects/09b75417dfc349edac1cb4a02522a204` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.604917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31982a-1fb1-4ed8-bd88-aa741a9c82ec None admin] RBAC: Authorizing `identity:get_project(project_id=09b75417dfc349edac1cb4a02522a204)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.606339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31982a-1fb1-4ed8-bd88-aa741a9c82ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.607936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 575/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/projects/09b75417dfc349edac1cb4a02522a204 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:06.614295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:06.663359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:06.724090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ba15a-d524-4376-9c9e-4c308891b55e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:06.763160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:06.763764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:06.764047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:06.764311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:06.765072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=09b75417dfc349edac1cb4a02522a204, user.id=ab9a96daebf94f2780001fe4b0e74fd4) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:06.765387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:06.766829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bde378-c99d-486d-a02a-ae79feda93b9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:06.772860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 571/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:06 2022] GET /identity/v3/role_assignments?user.id=ab9a96daebf94f2780001fe4b0e74fd4&scope.project.id=09b75417dfc349edac1cb4a02522a204&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:08.641195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde3fb0-b3e0-444f-974a-d5ca61df071a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.641788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde3fb0-b3e0-444f-974a-d5ca61df071a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.641788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde3fb0-b3e0-444f-974a-d5ca61df071a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.643065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 576/1147] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:08.648380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb332-580f-476a-955f-6825707a3fee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:08.648842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb332-580f-476a-955f-6825707a3fee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:08.648943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cb332-580f-476a-955f-6825707a3fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:08.666907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-733cb332-580f-476a-955f-6825707a3fee None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:08.668081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-733cb332-580f-476a-955f-6825707a3fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:08.733044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 572/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:08.998602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde3fb0-b3e0-444f-974a-d5ca61df071a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.076762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.099691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde3fb0-b3e0-444f-974a-d5ca61df071a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.133879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.134494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.134769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.135064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.144934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:09.149663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:09.150783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:27:09.152230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 577/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:09.157019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cb332-580f-476a-955f-6825707a3fee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.201089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.252264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cb332-580f-476a-955f-6825707a3fee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.285379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.285959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.286230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.286493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.287143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:09.287437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.288832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.296454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 573/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.302237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.419988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8766b36-c4cb-428f-be8c-f59dcd3728f8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.452717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.453327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.453601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.453871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] PATH_INFO: `/v3/users/ab9a96daebf94f2780001fe4b0e74fd4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.454963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] RBAC: Authorizing `identity:get_user(user_id=ab9a96daebf94f2780001fe4b0e74fd4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.457075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.458636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 578/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/users/ab9a96daebf94f2780001fe4b0e74fd4 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.464419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.560553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fc1e2-d800-40f4-b87d-da417136285a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.567717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.594230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e08ca-6d8a-4087-b644-1c9b8aa2583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.594845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e08ca-6d8a-4087-b644-1c9b8aa2583c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.595116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e08ca-6d8a-4087-b644-1c9b8aa2583c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.595381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e08ca-6d8a-4087-b644-1c9b8aa2583c None admin] PATH_INFO: `/v3/projects/09b75417dfc349edac1cb4a02522a204` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.596500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e08ca-6d8a-4087-b644-1c9b8aa2583c None admin] RBAC: Authorizing `identity:get_project(project_id=09b75417dfc349edac1cb4a02522a204)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.597985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e08ca-6d8a-4087-b644-1c9b8aa2583c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.599580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 574/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:09 2022] GET /identity/v3/projects/09b75417dfc349edac1cb4a02522a204 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:09.605456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:09.609178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:09.702420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b649a0-5b4a-46b2-bff0-340fbcfeec8d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:09.738404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25434f7-a53c-4a59-be4f-9acb888e8fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:09.739103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25434f7-a53c-4a59-be4f-9acb888e8fc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:09.739377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25434f7-a53c-4a59-be4f-9acb888e8fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:09.739647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25434f7-a53c-4a59-be4f-9acb888e8fc1 None admin] PATH_INFO: `/v3/projects/09b75417dfc349edac1cb4a02522a204/users/ab9a96daebf94f2780001fe4b0e74fd4/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:09.741612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25434f7-a53c-4a59-be4f-9acb888e8fc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=09b75417dfc349edac1cb4a02522a204, user_id=ab9a96daebf94f2780001fe4b0e74fd4, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:09.743037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25434f7-a53c-4a59-be4f-9acb888e8fc1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:09.825485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 579/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:09 2022] PUT /identity/v3/projects/09b75417dfc349edac1cb4a02522a204/users/ab9a96daebf94f2780001fe4b0e74fd4/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:11.710540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b4f9-9dc3-4aa2-ac20-bb3a961ef746 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.711072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b4f9-9dc3-4aa2-ac20-bb3a961ef746 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.711072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902b4f9-9dc3-4aa2-ac20-bb3a961ef746 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.712424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 575/1154] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:11.717614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:11.718045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:11.718168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:11.736605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:11.737902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:11.795102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:11.803165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 580/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:12.075876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b4f9-9dc3-4aa2-ac20-bb3a961ef746 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.177635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902b4f9-9dc3-4aa2-ac20-bb3a961ef746 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.210447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.211055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.211326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.211593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.221937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:12.226571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:12.227657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:27:12.228711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 576/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:12.233883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.329867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c662b-2690-4ae1-8b0d-6d8764c8cd77 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.361748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.362895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.363173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.363448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.364104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:12.364430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.365783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.373498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 581/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.380147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.414037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.485291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af5d09-d3fd-4b69-bcd9-eb8c398ef4b6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.517272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.517891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.518967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] PATH_INFO: `/v3/users/ab9a96daebf94f2780001fe4b0e74fd4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.520275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] RBAC: Authorizing `identity:get_user(user_id=ab9a96daebf94f2780001fe4b0e74fd4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.521755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.523326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 577/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/users/ab9a96daebf94f2780001fe4b0e74fd4 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.529804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.585587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.627481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6dffb-3017-4cc6-be16-89e858cdf173 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.660741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.661329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.661622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.661888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.726054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.727485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.786589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.829655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 582/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:12.835878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:12.933719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c7b05-b2d9-424a-b126-76db8b60a515 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:12.935205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:12.967181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c3d37-ec7d-49d7-8254-6cc538a8f0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:12.967901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c3d37-ec7d-49d7-8254-6cc538a8f0de None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:12.968179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c3d37-ec7d-49d7-8254-6cc538a8f0de None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:12.968472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c3d37-ec7d-49d7-8254-6cc538a8f0de None admin] PATH_INFO: `/v3/projects/09b75417dfc349edac1cb4a02522a204` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:12.969561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c3d37-ec7d-49d7-8254-6cc538a8f0de None admin] RBAC: Authorizing `identity:get_project(project_id=09b75417dfc349edac1cb4a02522a204)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:12.971005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c3d37-ec7d-49d7-8254-6cc538a8f0de None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:12.972679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 578/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/projects/09b75417dfc349edac1cb4a02522a204 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:12.978913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:13.077543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6471-cdca-4ff0-bfe7-2f9dc9276a4a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:13.115449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:13.115962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:13.116238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:13.116539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:13.117254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=09b75417dfc349edac1cb4a02522a204, user.id=ab9a96daebf94f2780001fe4b0e74fd4) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:13.117557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:13.119079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb2a95-517c-49b1-a446-46caa3d7616c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:13.124743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 583/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:12 2022] GET /identity/v3/role_assignments?user.id=ab9a96daebf94f2780001fe4b0e74fd4&scope.project.id=09b75417dfc349edac1cb4a02522a204&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:15.043865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b385-f881-46f5-990b-bd659d4740ba None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.044405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b385-f881-46f5-990b-bd659d4740ba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.044405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b385-f881-46f5-990b-bd659d4740ba None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.045746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 579/1162] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:15.051059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.051501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.051595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.069528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:15.070691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:15.086328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.139385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 584/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:15.414064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76b385-f881-46f5-990b-bd659d4740ba None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.514648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76b385-f881-46f5-990b-bd659d4740ba None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.547084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.547671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.547946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.548212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.585095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.612252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.614326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.710756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 580/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:15.716672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.814462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd6569-0964-4c67-a174-3951604826b1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:15.849075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:15.849686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:15.849972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:15.850241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:15.856685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:15.858109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-be7fab83-2c95-4dba-91e7-5ba2e43e411d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:15.869617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:15.870817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:27:15.871892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 585/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:15.876639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:15.946312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:15.975605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0bb54-df0b-47ca-86f3-cf718b15edff None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:16.009721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:16.010319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:16.010594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:16.010861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:16.011496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:16.011795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:16.013206 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271a043-b750-449c-b4dd-de668bd964e9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:16.020528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 581/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Aug 24 16:27:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:17.897676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7daae-7258-4f0c-ab37-9ca818d9bbff None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.898188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7daae-7258-4f0c-ab37-9ca818d9bbff None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.898188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7daae-7258-4f0c-ab37-9ca818d9bbff None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.899548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 586/1167] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:17.905124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bf064-36f0-4e7e-a9e8-0aa7baf2308c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:17.905511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bf064-36f0-4e7e-a9e8-0aa7baf2308c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:17.905666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bf064-36f0-4e7e-a9e8-0aa7baf2308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:17.925296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-654bf064-36f0-4e7e-a9e8-0aa7baf2308c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:17.927252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-654bf064-36f0-4e7e-a9e8-0aa7baf2308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:17.990685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 582/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:18.259163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7daae-7258-4f0c-ab37-9ca818d9bbff None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:18.272509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:18.358818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7daae-7258-4f0c-ab37-9ca818d9bbff None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:18.392018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc607b72-0de5-4401-a360-1e01cc6283ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:18.392630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc607b72-0de5-4401-a360-1e01cc6283ab None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:18.392910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc607b72-0de5-4401-a360-1e01cc6283ab None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:18.393180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc607b72-0de5-4401-a360-1e01cc6283ab None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:18.393841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc607b72-0de5-4401-a360-1e01cc6283ab None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:18.395436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc607b72-0de5-4401-a360-1e01cc6283ab None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:18.490071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 587/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Wed Aug 24 16:27:18 2022] POST /identity/v3/domains => generated 267 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:20.397266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb07b-09b3-4a8c-a5f7-d28b30e83afe None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.397801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb07b-09b3-4a8c-a5f7-d28b30e83afe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.397801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb07b-09b3-4a8c-a5f7-d28b30e83afe None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.399124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 583/1170] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:20.404589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.405022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.405131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.423484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:20.424702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:20.485330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.489904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 588/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:20.755947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb07b-09b3-4a8c-a5f7-d28b30e83afe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:20.818462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:20.855360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb07b-09b3-4a8c-a5f7-d28b30e83afe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:20.888493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf3d03-a259-4abe-a187-14222bf03623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:20.889127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf3d03-a259-4abe-a187-14222bf03623 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:20.889405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf3d03-a259-4abe-a187-14222bf03623 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:20.889678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf3d03-a259-4abe-a187-14222bf03623 None admin] PATH_INFO: `/v3/domains/60eebd89544d4b11967c62d899ca1948` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:20.900947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf3d03-a259-4abe-a187-14222bf03623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=60eebd89544d4b11967c62d899ca1948)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:20.902691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cf3d03-a259-4abe-a187-14222bf03623 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:20.904313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 584/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Aug 24 16:27:20 2022] GET /identity/v3/domains/60eebd89544d4b11967c62d899ca1948 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:20.910879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:21.007078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72077300-fd68-4eb8-99ca-f446cdb3704d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:21.040416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d4c07-95e9-490a-855a-cfbedf4cf5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:21.041009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d4c07-95e9-490a-855a-cfbedf4cf5ac None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:21.041290 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d4c07-95e9-490a-855a-cfbedf4cf5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:21.041564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d4c07-95e9-490a-855a-cfbedf4cf5ac None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:21.042212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d4c07-95e9-490a-855a-cfbedf4cf5ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:21.043656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d4c07-95e9-490a-855a-cfbedf4cf5ac None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:21.140278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 589/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:27:20 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:27:23.052620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d99fc-c4dc-4b65-a420-e83c08a96f4c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.053208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d99fc-c4dc-4b65-a420-e83c08a96f4c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.053208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d99fc-c4dc-4b65-a420-e83c08a96f4c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.054469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 585/1174] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:23.059600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.059996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.060146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.077997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:23.079259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:23.126568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.145547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 590/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:23.421182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d99fc-c4dc-4b65-a420-e83c08a96f4c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.528692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d99fc-c4dc-4b65-a420-e83c08a96f4c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.551723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:23.562043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7181ca-5d28-4524-b3e9-5a40df8c127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.566639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7181ca-5d28-4524-b3e9-5a40df8c127e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.566639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7181ca-5d28-4524-b3e9-5a40df8c127e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.566639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7181ca-5d28-4524-b3e9-5a40df8c127e None admin] PATH_INFO: `/v3/domains/60eebd89544d4b11967c62d899ca1948` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.566639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7181ca-5d28-4524-b3e9-5a40df8c127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=60eebd89544d4b11967c62d899ca1948)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.566639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7181ca-5d28-4524-b3e9-5a40df8c127e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.568020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 586/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Aug 24 16:27:23 2022] GET /identity/v3/domains/60eebd89544d4b11967c62d899ca1948 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:23.573849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:23.669784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6ddb9-6d28-49a4-a70d-3c6d8fbff6df None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:23.703757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5086067-c6d9-4abe-b9f9-5a631f738e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:23.704291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5086067-c6d9-4abe-b9f9-5a631f738e57 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:23.704598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5086067-c6d9-4abe-b9f9-5a631f738e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:23.704870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5086067-c6d9-4abe-b9f9-5a631f738e57 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:23.705516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5086067-c6d9-4abe-b9f9-5a631f738e57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:23.707219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5086067-c6d9-4abe-b9f9-5a631f738e57 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:23.786422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 591/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:27:23 2022] POST /identity/v3/users => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:27:25.726002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede54d5e-7d47-4e9e-b83d-1564fee0b5c3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.726567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede54d5e-7d47-4e9e-b83d-1564fee0b5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.726567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede54d5e-7d47-4e9e-b83d-1564fee0b5c3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.727863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 587/1178] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:25.733086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:25.733486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:25.733626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:25.752028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:25.753931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:25.783960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:25.821203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 592/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:27:26.089297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede54d5e-7d47-4e9e-b83d-1564fee0b5c3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.241799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede54d5e-7d47-4e9e-b83d-1564fee0b5c3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.274570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.275179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.275453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.275719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.286234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:26.290812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:26.291973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:27:26.293084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 588/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:26.297876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.392296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e03e6-87e2-4add-a6fe-99e65ce196c4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.425763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.426341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.426613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.426878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.427529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:26.427823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.429194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.436897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 593/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:27:26.442742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.494576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.541803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190925b-0601-4e48-af0c-71ce02068576 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.573543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.574151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.574421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.574684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] PATH_INFO: `/v3/users/f7e92ef8ed9d4ed3ad9fb2634059ceea` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.587875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] RBAC: Authorizing `identity:get_user(user_id=f7e92ef8ed9d4ed3ad9fb2634059ceea)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.589347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.590920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 589/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/users/f7e92ef8ed9d4ed3ad9fb2634059ceea => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:26.597619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.626997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.693712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66bd83-f554-4ec7-ab81-431885a2bcba None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:26.727342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:26.727927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:26.728199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:26.728491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:26.793138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:26.794577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:26.823259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:26.889592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 594/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:26.896321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:26.991599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea972af-661c-4921-a32f-61dcb84d3e18 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.001623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.025303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e411d8f-0248-47dd-b8af-541750ff5f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.026202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e411d8f-0248-47dd-b8af-541750ff5f70 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.026478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e411d8f-0248-47dd-b8af-541750ff5f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.026747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e411d8f-0248-47dd-b8af-541750ff5f70 None admin] PATH_INFO: `/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.027848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e411d8f-0248-47dd-b8af-541750ff5f70 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c6316e4a2f4ed693a1891c1b7fc74f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.029326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e411d8f-0248-47dd-b8af-541750ff5f70 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.030941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 590/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:26 2022] GET /identity/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:27.037109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:27.132596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e5267f-47e8-4a9f-8618-e2cd74d48049 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:27.159794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:27.165366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:27.165954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:27.166231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:27.166500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:27.167215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=f6c6316e4a2f4ed693a1891c1b7fc74f, user.id=f7e92ef8ed9d4ed3ad9fb2634059ceea) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:27.167519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:27.168966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4ec18-787f-404b-8d7f-3e7a2af495d3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:27.186884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 595/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:27 2022] GET /identity/v3/role_assignments?user.id=f7e92ef8ed9d4ed3ad9fb2634059ceea&scope.project.id=f6c6316e4a2f4ed693a1891c1b7fc74f&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.061754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7a7de-9403-4c63-b87c-c91d66abacb7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.062302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7a7de-9403-4c63-b87c-c91d66abacb7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.062302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7a7de-9403-4c63-b87c-c91d66abacb7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.063621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 591/1186] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:29.069408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.069827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.069931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.088134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:29.089336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:29.154339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 596/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:29.425977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7a7de-9403-4c63-b87c-c91d66abacb7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.524072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb7a7de-9403-4c63-b87c-c91d66abacb7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.557100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.557704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.557974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.558237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.568584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:29.573136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:29.574215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:27:29.575152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 592/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:29.580177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.676258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d196f20-b1af-430f-9cbb-b0457a91c42f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.698975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.709233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c8991-6756-4494-8d03-589c4467518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.709825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c8991-6756-4494-8d03-589c4467518b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.710101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c8991-6756-4494-8d03-589c4467518b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.710373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c8991-6756-4494-8d03-589c4467518b None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.711035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c8991-6756-4494-8d03-589c4467518b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:29.711337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c8991-6756-4494-8d03-589c4467518b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.712765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c8991-6756-4494-8d03-589c4467518b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.723887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 597/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.726142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.766776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:29.825513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516b85e-31e1-4bd0-924a-8cfc4e51cbfb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:29.858180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:29.858787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:29.859058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:29.859323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] PATH_INFO: `/v3/users/f7e92ef8ed9d4ed3ad9fb2634059ceea` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:29.860447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] RBAC: Authorizing `identity:get_user(user_id=f7e92ef8ed9d4ed3ad9fb2634059ceea)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:29.861883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:29.864037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 593/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/users/f7e92ef8ed9d4ed3ad9fb2634059ceea => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:29.869839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c8991-6756-4494-8d03-589c4467518b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:29.967163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c8991-6756-4494-8d03-589c4467518b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.007029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7acc6-f121-419f-a613-600c4adc3388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.007645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7acc6-f121-419f-a613-600c4adc3388 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.007920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7acc6-f121-419f-a613-600c4adc3388 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.008188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7acc6-f121-419f-a613-600c4adc3388 None admin] PATH_INFO: `/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.009438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7acc6-f121-419f-a613-600c4adc3388 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c6316e4a2f4ed693a1891c1b7fc74f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.010884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7acc6-f121-419f-a613-600c4adc3388 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.012553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 598/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:29 2022] GET /identity/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:30.018682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:30.116871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a44bfc-6ea5-4a37-a7e1-63c8961ae834 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:30.124378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:30.151729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647733b5-f288-4f88-a9c0-7d35ed26124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:30.152411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647733b5-f288-4f88-a9c0-7d35ed26124e None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:30.152690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647733b5-f288-4f88-a9c0-7d35ed26124e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:30.153005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647733b5-f288-4f88-a9c0-7d35ed26124e None admin] PATH_INFO: `/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f/users/f7e92ef8ed9d4ed3ad9fb2634059ceea/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:30.155360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647733b5-f288-4f88-a9c0-7d35ed26124e None admin] RBAC: Authorizing `identity:create_grant(project_id=f6c6316e4a2f4ed693a1891c1b7fc74f, user_id=f7e92ef8ed9d4ed3ad9fb2634059ceea, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:30.156829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647733b5-f288-4f88-a9c0-7d35ed26124e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:30.216295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 594/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:27:30 2022] PUT /identity/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f/users/f7e92ef8ed9d4ed3ad9fb2634059ceea/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:27:32.094370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f4656-6856-40a1-a29a-17b508aa66d2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.094886 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f4656-6856-40a1-a29a-17b508aa66d2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.094886 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f4656-6856-40a1-a29a-17b508aa66d2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.097040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 599/1193] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:27:32.102094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.102492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.102629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.120791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:27:32.121966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:27:32.186534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 595/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:27:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:27:32.452080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f4656-6856-40a1-a29a-17b508aa66d2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.507467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.551926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f4656-6856-40a1-a29a-17b508aa66d2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.583954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.584580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.584853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.585119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.595169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:27:32.600330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:27:32.601489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:27:32.602345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 600/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:27:32.607422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.698288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.703890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3fe80-d042-4f07-89c1-e765ca954523 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.737238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.737831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.738107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.738374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.739041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:32.739340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.740757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.748927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 596/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.755495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.853626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d6063-0e45-4c1e-b25e-2ca22418cccf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:32.866818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:32.886351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:32.886967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:32.887261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:32.887513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] PATH_INFO: `/v3/users/f7e92ef8ed9d4ed3ad9fb2634059ceea` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:32.888652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] RBAC: Authorizing `identity:get_user(user_id=f7e92ef8ed9d4ed3ad9fb2634059ceea)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:32.890066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:32.891639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 601/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/users/f7e92ef8ed9d4ed3ad9fb2634059ceea => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:32.898647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:32.995081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14fb5a-440e-4155-b5b0-6e09bde5578a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.028051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.028663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.028935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.029199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.054480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.093158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.094641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.190770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 597/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:27:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:27:33.196789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.296693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c229523-5d07-48a4-a8e6-84eb2962e7cc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.329475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd00f49-c075-4782-a1c9-3aae9f87edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.330086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00f49-c075-4782-a1c9-3aae9f87edb1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.330386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00f49-c075-4782-a1c9-3aae9f87edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.331160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd00f49-c075-4782-a1c9-3aae9f87edb1 None admin] PATH_INFO: `/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.332252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd00f49-c075-4782-a1c9-3aae9f87edb1 None admin] RBAC: Authorizing `identity:get_project(project_id=f6c6316e4a2f4ed693a1891c1b7fc74f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.333690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd00f49-c075-4782-a1c9-3aae9f87edb1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.335300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 602/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/projects/f6c6316e4a2f4ed693a1891c1b7fc74f => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:33.341431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:27:33.397393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:33.437818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e012ec-e7e1-47de-aab3-48aad776db37 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:27:33.471757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:27:33.472308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:27:33.472619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:27:33.472886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:27:33.473608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=f6c6316e4a2f4ed693a1891c1b7fc74f, user.id=f7e92ef8ed9d4ed3ad9fb2634059ceea) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:27:33.474011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:27:33.475452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac607c-2222-4c34-8c7b-e25894bbcc7f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:27:33.481096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 598/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:27:33 2022] GET /identity/v3/role_assignments?user.id=f7e92ef8ed9d4ed3ad9fb2634059ceea&scope.project.id=f6c6316e4a2f4ed693a1891c1b7fc74f&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:27:43.325377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=75 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:27:43.478526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=39 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:47.793868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8f35-3d7a-4928-b619-a4fe8eaa115a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.794476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8f35-3d7a-4928-b619-a4fe8eaa115a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.794476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8f35-3d7a-4928-b619-a4fe8eaa115a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.795826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 603/1201] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:47.800943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:47.801370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:47.801468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:47.819979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:47.821347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:47.887482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 599/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:48.159483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8f35-3d7a-4928-b619-a4fe8eaa115a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.260441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8f35-3d7a-4928-b619-a4fe8eaa115a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.294745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.295362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.295642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.295956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.306089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.307533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:48.327259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:48.328596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:48.329800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:48.329800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:48.329800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:28:48.329800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 604/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:48.335347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.435749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b043248-c1e6-4ff6-beaf-d5303fc78c1c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.463856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.469438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.470091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.470363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.470626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.471260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:48.471555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.472963 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e9c9f-c17b-46f6-9a03-8c33ca5576df None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.484448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 600/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:48.491269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:48.545138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:48.590214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881023d-255b-41f7-a0b2-0024a6c1950a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:48.623535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc06e2-008b-4e65-8a1b-1df68515a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:48.624123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc06e2-008b-4e65-8a1b-1df68515a56b None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:48.624434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc06e2-008b-4e65-8a1b-1df68515a56b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:48.624705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc06e2-008b-4e65-8a1b-1df68515a56b None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:48.625340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc06e2-008b-4e65-8a1b-1df68515a56b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:48.626772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc06e2-008b-4e65-8a1b-1df68515a56b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:48.725732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 605/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Aug 24 16:28:48 2022] POST /identity/v3/users => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:28:50.656542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499821-d4f8-46c8-9e7a-95ffff6455c2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.657111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499821-d4f8-46c8-9e7a-95ffff6455c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.657111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499821-d4f8-46c8-9e7a-95ffff6455c2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.658425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 601/1206] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:50.663559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:50.667254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:50.667254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:50.681918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:50.683114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:50.749031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 606/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:51.022055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499821-d4f8-46c8-9e7a-95ffff6455c2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.125023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499821-d4f8-46c8-9e7a-95ffff6455c2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.159208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.159831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.160108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.160407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.171466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:51.176183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:51.178002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:51.178896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 602/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:51.184115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.227959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:51.285328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77049e96-eb5b-492c-9afb-f51e7cd8f456 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.319704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.320300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.320601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.320868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.321526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:51.321820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.323254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.330950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 607/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:51.337473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.387444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:51.437009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a166da2-8465-44b5-9ff6-113ffa087411 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.472507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.473125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.473946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.474311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.485682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.487129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:51.501440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:51.502815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:51.504021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:51.504021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:51.504021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:51.504021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 603/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:51.508591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.604753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:51.607478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdc30-a204-4fa5-bd1e-d5c15add43eb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.640299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.640920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.641222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.641465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.642096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:51.642396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.643744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.655441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 608/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:51.661764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.761641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf67de-cfc6-47d9-af38-f799a85ff561 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.763432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:51.796069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.796715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.796988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.797251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.805145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.806625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:51.818326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:51.819333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:51.820319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 604/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:51.825644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:51.929094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f4442-4422-40e9-8a33-2ab61440465c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:51.964168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:51.964800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:51.965079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:51.965349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:51.966072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:51.966374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:51.967790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:51.985063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 609/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:51.991427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.092612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162bb9b-30cd-4912-a6e6-c380669ba225 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.124892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.125487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.125757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.126018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.156530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.193222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.194785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.296686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 605/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:28:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:28:52.303074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.321410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.401049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75719c7a-ab14-483f-9d60-640b48cec90e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.437074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.438069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.438577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.439063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.452478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.453917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:52.468465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:52.469588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:52.470750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:52.470750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:52.470750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:28:52.472487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 610/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:52.475917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.539956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.577505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211918b8-7c22-4743-aba6-1e2b86131541 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.611862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.612559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.612834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.613101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.613732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:52.614164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.615958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.626937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 606/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:52.632971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.700435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.731187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe09471-0173-4a18-b2ce-7b8fed376826 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.763647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.764257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.764562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.764832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.771755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.773457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:52.784280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:52.785351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:28:52.787251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 611/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:52.791883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:52.891335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de451f-e623-4795-9c94-7851261ca5a5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:52.914692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:52.925984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:52.926796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:52.927068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:52.927335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:52.928018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:52.928317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:52.929771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3435be-d364-4ebb-b16c-91a66601774b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:52.941291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 607/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:52.947893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:53.047742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af101b0-06f6-48b9-b174-d00f904289e2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:53.075976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:53.081712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:53.082310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:53.082584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:53.083456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:53.084174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=0234ca49360c4a7cbbaec848f27b89ad) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:53.084513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:53.085943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcba27-68b5-4142-8718-b9695a48ece4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:53.091228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 612/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:28:52 2022] GET /identity/v3/role_assignments?user.id=0234ca49360c4a7cbbaec848f27b89ad&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:54.995065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7ee58-c49a-4b8f-8b08-64f2a01ae687 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:54.995698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7ee58-c49a-4b8f-8b08-64f2a01ae687 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:54.995698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7ee58-c49a-4b8f-8b08-64f2a01ae687 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:54.996951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 608/1220] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:55.002016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.002420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.002563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.021077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:55.022616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:55.089491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 613/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:55.359010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7ee58-c49a-4b8f-8b08-64f2a01ae687 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.459115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7ee58-c49a-4b8f-8b08-64f2a01ae687 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.492507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.493123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.493392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.493654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.503480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:55.508447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:55.509468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:55.510398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 609/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:55.516032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.614950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd49f38-a48a-4e37-8e7d-7811ef2c1277 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.637266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.648499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.649128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.649404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.649674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.650335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:55.650634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.651998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.659622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 614/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:55.665940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.709154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:55.763107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c1a1e-ef66-4c8d-a0af-b63bd52a6770 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.797674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.798360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.798632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.798967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.809337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.810817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:55.825734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:55.826891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:55.828074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:55.828074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:55.828074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:55.828074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 610/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:55.833255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:55.934584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f343a5d-f985-45b0-947b-123ab9c5205f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:55.969089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:55.969689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:55.969965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:55.970231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:55.970857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:55.971152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:55.972547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:55.983943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 615/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:55.990168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.085253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.091436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4dddb-5073-4eea-a5be-ec170140a482 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.125214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.125867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.126160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.126431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.135249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.136795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:28:56.148872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:56.151913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:56.153172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 611/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:56.155984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.202580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.255851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534a9eb-61ad-41cd-b3ab-92325907e70f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.290088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.290687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.290966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.291236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.292043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:56.292374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.293902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.311393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 616/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:56.317863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.417119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f3075-c044-4ce3-99be-8d4b75a45714 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.450934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.451553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.451828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.452104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.462443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.463838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:56.477836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:56.478998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:56.480275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:56.480275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:56.480275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:56.480275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 612/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:56.485081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.595209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288d456-3a27-4d89-8996-5bcb5e72e75e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.597517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.628832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.629498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.629757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.630024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.630654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:56.630954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.632430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.643015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 617/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:56.649044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.658277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:56.746679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c638b-7c1e-4090-a7f4-8a3c6edc37cf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.779102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.779722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.779996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.780265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.787249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.788728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:28:56.799634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:56.800722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:28:56.802271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 613/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:56.807255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:56.914315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9cabe-a63f-43e7-9e6b-aa15ea93bf48 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:56.949319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:56.949918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:56.950196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:56.950473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:56.951157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:56.951463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:56.952967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75425a5-fe45-41ef-a951-6ce046050758 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:56.965142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 618/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:28:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:56.976488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:57.038246 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:57.073401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfa39-86a7-46e6-9615-e3e465a3947c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:57.106146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d87988-4ce6-447d-99eb-2c178f6f6796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:57.106792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d87988-4ce6-447d-99eb-2c178f6f6796 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:57.107068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d87988-4ce6-447d-99eb-2c178f6f6796 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:57.107336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d87988-4ce6-447d-99eb-2c178f6f6796 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/0234ca49360c4a7cbbaec848f27b89ad/roles/461ad5987b9342db89642171ce8457b1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:57.122061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d87988-4ce6-447d-99eb-2c178f6f6796 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=0234ca49360c4a7cbbaec848f27b89ad, role_id=461ad5987b9342db89642171ce8457b1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:57.123515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d87988-4ce6-447d-99eb-2c178f6f6796 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:57.188196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 614/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:28:56 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/0234ca49360c4a7cbbaec848f27b89ad/roles/461ad5987b9342db89642171ce8457b1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:28:59.077977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2c49b-8c6d-451d-a6ae-ac0041779951 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.078499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2c49b-8c6d-451d-a6ae-ac0041779951 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.078623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2c49b-8c6d-451d-a6ae-ac0041779951 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.079912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 619/1233] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:28:59.085701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.086139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.086243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.104020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:28:59.105318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:28:59.172725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 615/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:28:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:28:59.437227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2c49b-8c6d-451d-a6ae-ac0041779951 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.462859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.539910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2c49b-8c6d-451d-a6ae-ac0041779951 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.574677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.575344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.575617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.575884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.585830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:28:59.590501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:59.593083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:28:59.594123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 620/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:28:59.598423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.643878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.696615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9195-83b1-4bfa-b75d-af4c165096b6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.730210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.730844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.731121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.731417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.732199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:28:59.732600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:59.733993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:59.742939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 616/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:28:59.748815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:28:59.837656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:28:59.853287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032bb62a-7cb8-4ac3-84b8-fceaf1853eaa None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:28:59.885866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:28:59.886484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:28:59.886755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:28:59.887047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:28:59.898042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:28:59.899588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:28:59.915021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:28:59.916859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:28:59.917967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:28:59.917967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:28:59.917967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:28:59.917967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 621/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:28:59.924270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.026322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368243c0-0509-4400-9b30-2d76d353e15a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.034979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.059459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.060052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.060326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.060622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.061255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:00.061551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.063021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.074332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 617/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:00.080063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.179395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa93642-9067-48f3-867c-1c4f4b115779 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.213057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.213677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.213956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.214225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.216881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.221852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.223910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:00.234836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:00.238683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:00.239623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 622/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:00.241947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.343462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb1f4-9293-4a73-9c5f-338ff6917df1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.376392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.376986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.377257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.377520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.378237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:00.378532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.380576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.398194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 618/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:00.403744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.505030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638777-f4a0-4a1c-85f7-e065eb5a7657 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.539363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.539955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.540230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.540527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.601312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.607795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.609256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.716644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 623/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:00.722416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.750453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.822375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4d6c9-7098-4fe9-a007-7e7cf7235848 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:00.855988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:00.856575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:00.856850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:00.857118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:00.867025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:00.868518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:00.882914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:00.883879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:00.884868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:00.884868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:00.884868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:00.884868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 619/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:00.890414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:00.985235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:00.991633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0f2db-4aa4-4069-ba56-ee9ce9b773a9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.032050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.032687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.032969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.033241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.033880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:01.034180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.036124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.047342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 624/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:01.054137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.131372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:01.152102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff25181-116e-4dce-b478-bcd3c3f660bf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.185606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.186227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.186505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.186772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.193364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.194824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:01.205996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:01.207141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:01.208237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 620/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:01.213186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.314804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aeafa-59fe-4bab-b666-9932dfc0e432 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.356792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.357395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.357668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.357932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.358615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:01.358946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.360508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e379d9-8971-43fc-8899-d4e96486d155 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.372995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 625/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:01.379802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:01.483300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c8da1-d293-4692-bfb1-10ec911279fe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:01.516893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:01.517500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:01.517754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:01.518014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:01.518730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] RBAC: Adding query filter params (role.id=461ad5987b9342db89642171ce8457b1, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=0234ca49360c4a7cbbaec848f27b89ad) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:01.519033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:01.520908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f94ad0-3ce9-407f-a589-cab199e3c5ca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:01.526854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 621/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:01 2022] GET /identity/v3/role_assignments?user.id=0234ca49360c4a7cbbaec848f27b89ad&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=461ad5987b9342db89642171ce8457b1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:03.501125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131721e-d8ca-433b-8829-77fd55ae3bb9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.501680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131721e-d8ca-433b-8829-77fd55ae3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.501680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131721e-d8ca-433b-8829-77fd55ae3bb9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.503035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 626/1247] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:03.508998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:03.509383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:03.509535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:03.515819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.528689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:03.529863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:03.594993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 622/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:03.874417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131721e-d8ca-433b-8829-77fd55ae3bb9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:03.892461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:03.978135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131721e-d8ca-433b-8829-77fd55ae3bb9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.011005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.011614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.011892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.012158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.022925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:04.027560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.028796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:04.030056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 627/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.034687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.137466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561bdf7-3f75-46ee-aa71-8f2cc263df9e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.166604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.171428 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.172007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.172275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.172572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.173228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.173523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.174898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.182672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 623/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.188538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.265753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.288669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858a762-81e4-48ff-a40e-d44c0870b384 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.322619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.323230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.323504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.323769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.334481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.335903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:04.350694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.351909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:04.353258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:04.353258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:04.353258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:04.353258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 628/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.357836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.455555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3f433-847a-4f9c-b58a-71b72b84d24b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.489350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.489953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.490229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.490497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.491140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.491438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.492864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.505736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 624/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:04.511386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.614444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687b7dd-e888-439a-82bb-bd5e529b00eb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.640592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.648505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.649119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.649395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.649667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.657319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.658768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:04.670113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:04.671212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:04.672277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 629/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:04.677890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.732390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:04.776541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dac81-a184-4845-98a8-c5f6aa428961 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.810877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.811545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.811823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.812088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:04.812834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:04.813134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:04.814565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:04.831273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 625/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:04.837202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:04.938268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277510-577a-4478-92d7-f7d4efd95047 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:04.973447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:04.974174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:04.974451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:04.974844 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.032105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.047094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.048501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.181158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 630/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:05.187114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.287189 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c44c4-7021-4ee5-ab72-99985ea7da47 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.299694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.321037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.321659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.321934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.322321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.332332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.334127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:05.349190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:05.353609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:05.354708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:05.354708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:05.354708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:05.354708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 626/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:05.356179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.442459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.454217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc8552-1c3f-4b12-b8e4-8cdcbb06e2c7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.486805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.487395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.487674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.488615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.489246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:05.489544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.490892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.502391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 631/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:05.508831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.607439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85eea2-7b8e-4aa4-9feb-7015c53c2af0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.641343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.641965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.642242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.642511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.650098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.656298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:05.662855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:05.663894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:05.665043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 627/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:05.669916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.769869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8920c0-7175-450d-9b38-aeb85fab1a28 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.804167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.804794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.805070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.805338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.806020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:05.806320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.807731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d4d19-5f5d-490c-9a9c-1d4fe9fb299c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.819906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 632/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:05.826133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:05.852475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:05.930115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8ef6e-5e7f-4c7a-b63d-ef280303c5cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:05.963379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:05.963997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:05.964268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:05.964562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:05.965279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=0234ca49360c4a7cbbaec848f27b89ad) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:05.965579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:05.967009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9913b6-86b6-4de8-910e-93b05352ad78 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:05.973998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 628/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:05 2022] GET /identity/v3/role_assignments?user.id=0234ca49360c4a7cbbaec848f27b89ad&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:07.871670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855fad3-a2c6-4772-af39-0f175fb9e160 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.872208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855fad3-a2c6-4772-af39-0f175fb9e160 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.872208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855fad3-a2c6-4772-af39-0f175fb9e160 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.873584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 633/1261] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:07.879534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:07.879978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:07.880066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:07.902867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:07.904057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:07.971248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 629/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:08.240853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855fad3-a2c6-4772-af39-0f175fb9e160 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.251704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.337533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855fad3-a2c6-4772-af39-0f175fb9e160 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.370934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.371544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.371817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.372081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.381941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:08.386563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:08.387703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:08.388899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 634/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:08.394229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.413273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.493127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf957f8-2512-49d5-abb3-5191c74e976c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.529552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.530158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.530433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.530704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.531371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:08.531673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.533108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.541821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 630/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:08.547772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.615929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.649153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abacfbe-a9eb-4714-9fc3-2c05a4ba0c9f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.681884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.682502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.682775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.683041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.694090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.695617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:08.709904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:08.717417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:08.718541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:08.718541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:08.718541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:08.718541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 635/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:08.718816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.804197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:08.822709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b42c52-900a-4bba-9334-ce3b161e1b44 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.855924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:08.856573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:08.856848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:08.857116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:08.857760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:08.858058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:08.859437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:08.871535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 631/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:08.877284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:08.979979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213bdd8-9efa-4121-8f98-b55b0e6b52a7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:08.999581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.013859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.014491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.014767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.015033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.024929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.026554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:09.038166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:09.039278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:09.040463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 636/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:09.045286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.149767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44ad14-d672-46ca-b602-1b7c71cfc07b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.185398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.186029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.186363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.186738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.187478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:09.187781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.189244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.199170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.207511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 632/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:09.213240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.314676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8c3bb-b8dc-4bc2-b539-4ec39d27b09b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.348716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.349365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.349640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.349904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.360935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.365092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:09.376750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:09.377901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:09.379005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:09.379005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:09.379005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:09.379005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 637/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:09.383746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.483488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896943b-0295-4794-98de-f04ee1f5fd5e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.517588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.518197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.518473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.518969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.519620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:09.519913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.521334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.532988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 633/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:09.539723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.572193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.639368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70445945-d613-4d35-88a3-92de4bfc3593 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.673875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.674491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.674767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.675633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.682098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.683523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:09.694899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:09.695950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:09.697096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 638/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:09.701818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.760274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.801878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea272f09-76d6-4410-9da5-57f077f73e0f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.835503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.836089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.836384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.836656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.837370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:09.837664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:09.839085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6272926-0432-4393-a8e1-28560a97e174 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:09.851083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 634/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:09.857151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:09.952843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:09.961877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bba72b-2e76-4bed-a904-0308d6698208 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:09.995988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382522e-347d-43ac-aa21-5ed411232d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:09.996673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382522e-347d-43ac-aa21-5ed411232d10 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:09.996951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382522e-347d-43ac-aa21-5ed411232d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:09.997216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382522e-347d-43ac-aa21-5ed411232d10 None admin] PATH_INFO: `/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/0234ca49360c4a7cbbaec848f27b89ad/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:09.999230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382522e-347d-43ac-aa21-5ed411232d10 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed3e2826fd1c4c16a6a99d0fc206b22e, user_id=0234ca49360c4a7cbbaec848f27b89ad, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:10.000685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382522e-347d-43ac-aa21-5ed411232d10 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:10.133095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 639/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Aug 24 16:29:09 2022] PUT /identity/v3/projects/ed3e2826fd1c4c16a6a99d0fc206b22e/users/0234ca49360c4a7cbbaec848f27b89ad/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:29:12.047866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508ef9-bb8c-4c9a-af19-90c912247711 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.048476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508ef9-bb8c-4c9a-af19-90c912247711 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.048572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508ef9-bb8c-4c9a-af19-90c912247711 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.050666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 635/1274] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:12.055901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.056279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.056475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.074810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:12.075996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:12.141892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 640/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:12.411074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30508ef9-bb8c-4c9a-af19-90c912247711 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:12.516155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30508ef9-bb8c-4c9a-af19-90c912247711 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:12.535215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:12.552278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:12.552921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.553194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.553460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.563372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 16:29:12.568775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:12.569905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:12.570899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 636/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Aug 24 16:29:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:12.575851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:12.664671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:12.682274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967ab4a-cdef-4996-be50-a17382c26160 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:12.714782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:12.715376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.715656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.715924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.716607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:12.716909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:12.718253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:12.726330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 641/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Aug 24 16:29:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:12.732164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:12.833461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cdae7-1eb6-4d72-a6e8-5d198419a851 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:12.866781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:12.867400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:12.867675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:12.867943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:12.879420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:12.880929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:12.895308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:12.897026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:12.898002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:12.898002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:12.898002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:12.898002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 637/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:12.902739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.003291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bbb4e-b9a1-4205-9769-cb7606171b12 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.036851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.037445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.037721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.037989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.038622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.038922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.040334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.051827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 642/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:13.057751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.089826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.158452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ea368-7765-4391-863f-622ee33ca0e2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.192838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.193452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.193721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.193982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.201161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.202608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 16:29:13.213948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:13.225002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:13.226092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 638/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:13.226694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.226694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.318495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd7d9-3362-454a-a629-5cf00339de6d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.350452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.351038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.351312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.351576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.352284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:13.352610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.354015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.372109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 643/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:13.378776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.473600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.483019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b8458-91bb-41c8-957d-762d286503f3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.516007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.516635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.516913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.517218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.584843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.586222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.671290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.681693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 639/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 16:29:13.687804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.762400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:13.800517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41a44a-e81a-4865-be5f-f1d691838b03 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:13.832944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:13.833727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:13.834004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:13.834272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:13.845351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:13.847278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 16:29:13.861186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:13.866613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 16:29:13.867770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 16:29:13.867770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 16:29:13.867770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:13.867770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 644/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 16:29:13.868751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:13.973130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9afefd-3bff-49d9-9b68-8244a13f6ad9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.007547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.008194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.008508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.008782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.009431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:14.009734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.011181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.023573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 640/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Aug 24 16:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:14.029443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.136768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d3601-592f-4834-9e23-f22d7f6fb803 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.151189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.170197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.170810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.171108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.171370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.178336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.179763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 16:29:14.191677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:14.192756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 24 16:29:14.193871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 645/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:14.198911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.248716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:14.299302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e99a8c-0304-4ce3-8673-1b79e34141da None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.332792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.333393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.333664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.333982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.334669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:14.334966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.336425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09124d-9fda-4337-9a02-ab94767d61ea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.348830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 641/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:14.355213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:14.462942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f21f-0a92-415b-8487-0c9044190b3f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:14.507291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:14.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:14.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:14.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:14.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] RBAC: Adding query filter params (role.id=20693bab78b1461298baee2d4c8e1f27, scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e, user.id=0234ca49360c4a7cbbaec848f27b89ad) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:14.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:14.511620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c952-b23e-4d2f-9f89-699f9d9a15ba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:14.528259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 646/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Aug 24 16:29:14 2022] GET /identity/v3/role_assignments?user.id=0234ca49360c4a7cbbaec848f27b89ad&scope.project.id=ed3e2826fd1c4c16a6a99d0fc206b22e&role.id=20693bab78b1461298baee2d4c8e1f27 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:29:16.507274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31ea0b-64fa-4507-8c55-9a85ed6e9eaa None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.507832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31ea0b-64fa-4507-8c55-9a85ed6e9eaa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.507927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31ea0b-64fa-4507-8c55-9a85ed6e9eaa None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.509313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 642/1288] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:16.514656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:16.515104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:16.515204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:16.534365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:16.535540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:16.563626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:16.601527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 647/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:16.872992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31ea0b-64fa-4507-8c55-9a85ed6e9eaa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:16.976833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31ea0b-64fa-4507-8c55-9a85ed6e9eaa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:16.987380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:17.010372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.010989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.011265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.011534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.012110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.013529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:17.022936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:17.024179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:17.025267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 643/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:17.029997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.129769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f620b-4470-4e26-b8e8-7ddf97d88bc4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.169396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.170006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.170370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.170639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.171269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:17.171647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.173053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa930858-e6f3-429a-a72b-eababb217747 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.179762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 648/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Aug 24 16:29:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:17.186018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:17.286277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37e47a-1006-460f-a505-2fb969549d0b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:17.319930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:17.320551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:17.320823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:17.321085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:17.321702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:17.321996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:17.323364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ccded-3768-4f70-bdbc-9873a936113a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:17.327632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 644/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Aug 24 16:29:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:19.204805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7a0d-c23a-4382-9927-4b2d5abb2732 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.205867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7a0d-c23a-4382-9927-4b2d5abb2732 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.206008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b7a0d-c23a-4382-9927-4b2d5abb2732 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.207319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 649/1293] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:19.212483 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df7d29-c343-47b3-a1fb-c819b906f8d5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.212868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df7d29-c343-47b3-a1fb-c819b906f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.213029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df7d29-c343-47b3-a1fb-c819b906f8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.231114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b5df7d29-c343-47b3-a1fb-c819b906f8d5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:19.232297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b5df7d29-c343-47b3-a1fb-c819b906f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:19.261807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:19.297754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 645/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 16:29:19.568604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7a0d-c23a-4382-9927-4b2d5abb2732 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:19.599024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:19.668508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b7a0d-c23a-4382-9927-4b2d5abb2732 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:19.702339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4938d-2a04-4da9-8e33-8d265ed5a1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:19.702931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4938d-2a04-4da9-8e33-8d265ed5a1c5 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:19.703207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4938d-2a04-4da9-8e33-8d265ed5a1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:19.703476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4938d-2a04-4da9-8e33-8d265ed5a1c5 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:19.704117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4938d-2a04-4da9-8e33-8d265ed5a1c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:19.705892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4938d-2a04-4da9-8e33-8d265ed5a1c5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:19.829494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 650/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Aug 24 16:29:19 2022] POST /identity/v3/services => generated 246 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:29:21.766032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c202-77ce-45d0-9f5c-1ee3c1e7970e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.766563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c202-77ce-45d0-9f5c-1ee3c1e7970e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.766563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c202-77ce-45d0-9f5c-1ee3c1e7970e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.768791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 646/1296] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:21.774385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:21.774792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:21.774933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:21.793534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:21.794717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:21.867281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 651/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:22.138413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c202-77ce-45d0-9f5c-1ee3c1e7970e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:22.243271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c202-77ce-45d0-9f5c-1ee3c1e7970e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:22.278767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:22.279401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.279660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.279924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.280528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:22.281918 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:22.290586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:22.291631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:22.292533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 647/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:22.297953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:22.316288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:22.398305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfc519-2553-4343-ac2b-0ffb05a4f9b4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:22.431538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:22.432128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.432438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.432707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.433390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:22.433715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:22.435082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:22.439998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 652/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Aug 24 16:29:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:22.446153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:22.479005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:22.548309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645460e-3683-412d-ae4f-96feb3c63ecc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:22.580992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:22.581492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.581819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.582049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.582741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] RBAC: Adding query filter params (interface=public, service_id=f9c1b9845add4bce93ed574334ca3891, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:22.583038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:22.584437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dfefd1-79cf-43a1-aead-b88328fad809 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:22.589861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 648/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Aug 24 16:29:22 2022] GET /identity/v3/endpoints?service_id=f9c1b9845add4bce93ed574334ca3891&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:22.596209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:22.677903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:22.695610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0ceef-9e10-4cbc-9683-c0b298e13725 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:22.729923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62184a-06e6-47e6-9aa6-bdb7c875d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:22.730524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62184a-06e6-47e6-9aa6-bdb7c875d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:22.730801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62184a-06e6-47e6-9aa6-bdb7c875d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:22.731068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62184a-06e6-47e6-9aa6-bdb7c875d2c7 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:22.731666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62184a-06e6-47e6-9aa6-bdb7c875d2c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:22.733074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62184a-06e6-47e6-9aa6-bdb7c875d2c7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:22.740174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 653/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Aug 24 16:29:22 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:29:24.651015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9371bf-e346-44d2-b3e6-1c68c057ea7d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.651543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9371bf-e346-44d2-b3e6-1c68c057ea7d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.651543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9371bf-e346-44d2-b3e6-1c68c057ea7d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.652923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 649/1302] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:29:24.658219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:24.658586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:24.658806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:24.677082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:29:24.678270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:29:24.744572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 654/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:29:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 16:29:25.018899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9371bf-e346-44d2-b3e6-1c68c057ea7d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.113308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:25.119897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9371bf-e346-44d2-b3e6-1c68c057ea7d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.152472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.153089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.153363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.153630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.154200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.155578 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 16:29:25.165755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 16:29:25.167085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Aug 24 16:29:25.167969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 650/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 16:29:25.172695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.248531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:25.270800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da8c43-5f08-4ea8-b183-01f331442347 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.304933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.305534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.305814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.306114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.306738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:29:25.307037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.308409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6360f7e-55d6-4a40-a233-fb82206f4c04 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.313712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 655/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Aug 24 16:29:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:29:25.319815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:29:25.430549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208de36-7fa9-4c4b-850b-98f640776579 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:29:25.462790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aacd7-5273-42d0-a69f-58d2808eb2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:29:25.463384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aacd7-5273-42d0-a69f-58d2808eb2ea None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:29:25.463653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aacd7-5273-42d0-a69f-58d2808eb2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:29:25.463915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aacd7-5273-42d0-a69f-58d2808eb2ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:29:25.464590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aacd7-5273-42d0-a69f-58d2808eb2ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:29:25.465951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aacd7-5273-42d0-a69f-58d2808eb2ea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:29:25.588408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 651/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:29:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:29:35.378087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=30 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:29:35.583874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:05.100979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b2e5-f5e6-4ce2-a7f1-33991472e023 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:05.101497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b2e5-f5e6-4ce2-a7f1-33991472e023 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:05.101497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b2e5-f5e6-4ce2-a7f1-33991472e023 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:05.102854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 656/1307] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:05.108607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef6fda-ce1a-4dc3-ba28-5da359dbed96 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:05.109056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef6fda-ce1a-4dc3-ba28-5da359dbed96 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:05.109283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef6fda-ce1a-4dc3-ba28-5da359dbed96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:05.213071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3eef6fda-ce1a-4dc3-ba28-5da359dbed96 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:05.214275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3eef6fda-ce1a-4dc3-ba28-5da359dbed96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:05.396446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 652/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:05.731231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcf67e-0e90-4f3e-a385-2aae466e625b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:05.731509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcf67e-0e90-4f3e-a385-2aae466e625b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:05.731783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcf67e-0e90-4f3e-a385-2aae466e625b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:05.733606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 657/1309] 10.222.0.17 () {54 vars in 965 bytes} [Wed Aug 24 16:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:05.739711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6c39a-faad-47b7-a6ff-ed19b32b50be None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:05.740459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6c39a-faad-47b7-a6ff-ed19b32b50be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:05.740879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6c39a-faad-47b7-a6ff-ed19b32b50be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:05.825791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b2e6c39a-faad-47b7-a6ff-ed19b32b50be None None] MFA Rules not processed for user `9bde9ceb3c7c46298626580b62e0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:05.827228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b2e6c39a-faad-47b7-a6ff-ed19b32b50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:05.851899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:05.926051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 653/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Wed Aug 24 16:36:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:05.935058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcf67e-0e90-4f3e-a385-2aae466e625b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:05.937050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6cdcf67e-0e90-4f3e-a385-2aae466e625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:06.070789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcf67e-0e90-4f3e-a385-2aae466e625b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:06.158316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:06.159744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:06.160192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:06.160627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:06.162396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:06.225047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:06.285936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:06.287596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ad9d5-5eca-4278-a9c8-6e01ce4c544e service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:06.308621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 658/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:08.633394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a701-d190-419c-88c7-5ab2885682e8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:08.633873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a701-d190-419c-88c7-5ab2885682e8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:08.633998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52a701-d190-419c-88c7-5ab2885682e8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:08.635282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 654/1312] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:08.640665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c26b5-cf7a-4de1-9be5-eb8ed096bc37 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:08.641096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c26b5-cf7a-4de1-9be5-eb8ed096bc37 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:08.641221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c26b5-cf7a-4de1-9be5-eb8ed096bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:08.725101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-833c26b5-cf7a-4de1-9be5-eb8ed096bc37 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:08.726369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-833c26b5-cf7a-4de1-9be5-eb8ed096bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:08.828662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 659/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:09.139577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54ce7a-ae77-43f0-aa07-f07c1de264bb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:09.139883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54ce7a-ae77-43f0-aa07-f07c1de264bb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:09.140158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54ce7a-ae77-43f0-aa07-f07c1de264bb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:09.142104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 655/1314] 10.222.0.17 () {54 vars in 965 bytes} [Wed Aug 24 16:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:09.148253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9303-f50d-45d8-b4e0-7f63218107d2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:09.148576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9303-f50d-45d8-b4e0-7f63218107d2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:09.148853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac9303-f50d-45d8-b4e0-7f63218107d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:09.162304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:09.230671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c2ac9303-f50d-45d8-b4e0-7f63218107d2 None None] MFA Rules not processed for user `9bde9ceb3c7c46298626580b62e0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:09.231907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c2ac9303-f50d-45d8-b4e0-7f63218107d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:09.310357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 660/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Wed Aug 24 16:36:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:09.319922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54ce7a-ae77-43f0-aa07-f07c1de264bb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:09.321466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af54ce7a-ae77-43f0-aa07-f07c1de264bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:09.427485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54ce7a-ae77-43f0-aa07-f07c1de264bb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:09.496493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:09.499001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:09.499598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:09.499873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:09.500142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:09.502397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:09.614728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:09.616215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc17fe-ec3a-4851-92bc-ad1fea585ce5 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:09.634309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 656/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:12.265718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d81c6f-1fe9-4905-8651-956678889e6c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:12.266914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d81c6f-1fe9-4905-8651-956678889e6c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:12.266914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d81c6f-1fe9-4905-8651-956678889e6c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:12.268160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 661/1317] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:12.273667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c9ef-7c73-4059-aaff-721a860df03d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:12.274178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c9ef-7c73-4059-aaff-721a860df03d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:12.274276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c9ef-7c73-4059-aaff-721a860df03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:12.341949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a5c3c9ef-7c73-4059-aaff-721a860df03d None None] MFA Rules not processed for user `a94c4cb016bc4be3a26160a9d7f8b399`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:12.343149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5c3c9ef-7c73-4059-aaff-721a860df03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:12.395095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:12.439639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 657/1318] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:12.731505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d81c6f-1fe9-4905-8651-956678889e6c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:12.733085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-96d81c6f-1fe9-4905-8651-956678889e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:12.838826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d81c6f-1fe9-4905-8651-956678889e6c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:12.841685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:12.913518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:12.914139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:12.914417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:12.914687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:12.917098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:13.044145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:13.045623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1315d41-5be1-479e-9b7b-c25599a3a386 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:13.064690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 662/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:15.782780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a2860-12ab-4db3-bdec-3dfa277ed934 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:15.783339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a2860-12ab-4db3-bdec-3dfa277ed934 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:15.783339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a2860-12ab-4db3-bdec-3dfa277ed934 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:15.784742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 658/1320] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:15.789839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aacca1-d311-457f-b20a-115f17b6ea59 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:15.790240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aacca1-d311-457f-b20a-115f17b6ea59 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:15.790389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aacca1-d311-457f-b20a-115f17b6ea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:15.831459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:15.856581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-04aacca1-d311-457f-b20a-115f17b6ea59 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:15.857788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-04aacca1-d311-457f-b20a-115f17b6ea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:15.953285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 663/1321] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:16.235867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a2860-12ab-4db3-bdec-3dfa277ed934 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:16.237439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c46a2860-12ab-4db3-bdec-3dfa277ed934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:16.340605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a2860-12ab-4db3-bdec-3dfa277ed934 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:16.414586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:16.415187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:16.415464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:16.415729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:16.417359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:16.425213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:16.523593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:16.525095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40eb594-c996-4d2f-9838-34c0013f7c9d service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:16.543348 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 659/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:36:19.460820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932027b-7e4d-4ba4-97ad-74426c82ba9d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:19.461360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932027b-7e4d-4ba4-97ad-74426c82ba9d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:19.461449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932027b-7e4d-4ba4-97ad-74426c82ba9d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:19.462732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 664/1323] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:19.468205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449ed60-5236-4440-9bde-122d6c1577c6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:19.475195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449ed60-5236-4440-9bde-122d6c1577c6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:19.475195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449ed60-5236-4440-9bde-122d6c1577c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:19.538821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c449ed60-5236-4440-9bde-122d6c1577c6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:19.540029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c449ed60-5236-4440-9bde-122d6c1577c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:19.600411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:19.638594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 660/1324] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:19.921415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932027b-7e4d-4ba4-97ad-74426c82ba9d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:19.923132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0932027b-7e4d-4ba4-97ad-74426c82ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:20.034596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932027b-7e4d-4ba4-97ad-74426c82ba9d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:20.047914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:20.110610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:20.111217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:20.111498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:20.111763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:20.113415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:20.225788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:20.227193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba269f5-ef72-4d5b-a69c-0d0653be1366 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:20.246224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 665/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:22.222243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff9c53-c7bd-4c32-93b1-66945df7e2ca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:22.222761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff9c53-c7bd-4c32-93b1-66945df7e2ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:22.222761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff9c53-c7bd-4c32-93b1-66945df7e2ca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:22.224031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 661/1326] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:22.229293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc758c-82c7-4070-bd3f-ddb808c6e06a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:22.229720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc758c-82c7-4070-bd3f-ddb808c6e06a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:22.229930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc758c-82c7-4070-bd3f-ddb808c6e06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:22.278546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:22.296533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-89bc758c-82c7-4070-bd3f-ddb808c6e06a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:22.297857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-89bc758c-82c7-4070-bd3f-ddb808c6e06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:22.396246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 666/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:22.676303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff9c53-c7bd-4c32-93b1-66945df7e2ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:22.677896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e9ff9c53-c7bd-4c32-93b1-66945df7e2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:22.795205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff9c53-c7bd-4c32-93b1-66945df7e2ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:22.870349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:22.870954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:22.871230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:22.871551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:22.873114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:22.892788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:22.998003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:22.999464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463c2b7-dbf3-4f70-8848-0c8a08b15c86 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:23.024332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 662/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:25.009145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30080d97-ddd5-44b5-a472-34df859e81ce None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:25.009711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30080d97-ddd5-44b5-a472-34df859e81ce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:25.009711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30080d97-ddd5-44b5-a472-34df859e81ce None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:25.011043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 667/1329] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:25.016518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4ed5e-a0c0-4106-bfee-3163ac68330b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:25.016974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4ed5e-a0c0-4106-bfee-3163ac68330b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:25.017095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4ed5e-a0c0-4106-bfee-3163ac68330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:25.090067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e5d4ed5e-a0c0-4106-bfee-3163ac68330b None None] MFA Rules not processed for user `a94c4cb016bc4be3a26160a9d7f8b399`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:25.091768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5d4ed5e-a0c0-4106-bfee-3163ac68330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:25.165064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:25.197340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 663/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:25 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:25.482622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30080d97-ddd5-44b5-a472-34df859e81ce None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:25.484373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-30080d97-ddd5-44b5-a472-34df859e81ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:25.611235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30080d97-ddd5-44b5-a472-34df859e81ce None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:25.632144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:25.693430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:25.694036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:25.694313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:25.694578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:25.696314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:25.805695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:25.807072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f56c8-345f-42f1-9be4-b721616748cc service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:25.825507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 668/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:28.363696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064fc6f-46dc-45f2-88e3-67dd01afae21 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:28.364222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064fc6f-46dc-45f2-88e3-67dd01afae21 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:28.364222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064fc6f-46dc-45f2-88e3-67dd01afae21 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:28.365606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 664/1332] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:28.370745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82dd0b-4d58-480c-a0ca-b6564e8c127c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:28.371188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82dd0b-4d58-480c-a0ca-b6564e8c127c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:28.371285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82dd0b-4d58-480c-a0ca-b6564e8c127c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:28.435584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:28.439247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5a82dd0b-4d58-480c-a0ca-b6564e8c127c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:28.440694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5a82dd0b-4d58-480c-a0ca-b6564e8c127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:28.536731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 669/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:28.820123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064fc6f-46dc-45f2-88e3-67dd01afae21 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:28.821712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b064fc6f-46dc-45f2-88e3-67dd01afae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:28.933674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064fc6f-46dc-45f2-88e3-67dd01afae21 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:29.010572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:29.011264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:29.011463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:29.011735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:29.013391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:29.042957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:29.136640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:29.138197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865cde4-f525-4619-b90f-cf640e8ffcd1 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:29.160362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 665/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:31.126587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4646d77-00f2-492c-8ced-a6bbd3aca3ec None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:31.127103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4646d77-00f2-492c-8ced-a6bbd3aca3ec None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:31.127103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4646d77-00f2-492c-8ced-a6bbd3aca3ec None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:31.128406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 670/1335] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:31.133599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc6180-8753-470e-9c00-d9423c75347b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:31.133989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc6180-8753-470e-9c00-d9423c75347b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:31.134152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc6180-8753-470e-9c00-d9423c75347b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:31.200065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0adc6180-8753-470e-9c00-d9423c75347b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:31.201296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0adc6180-8753-470e-9c00-d9423c75347b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:31.274968 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:31.297688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 666/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:31.588122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4646d77-00f2-492c-8ced-a6bbd3aca3ec None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:31.589691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d4646d77-00f2-492c-8ced-a6bbd3aca3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:31.693652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4646d77-00f2-492c-8ced-a6bbd3aca3ec None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:31.721088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:31.769916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:31.770533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:31.770807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:31.771071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:31.773016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:31.883690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:31.885149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1486449-6645-406f-bf54-78692d1cc556 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:31.904641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 671/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:34.433897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517e8ed-09df-4025-815a-a5b92b4221a1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:34.434437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517e8ed-09df-4025-815a-a5b92b4221a1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:34.434437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517e8ed-09df-4025-815a-a5b92b4221a1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:34.435784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 667/1338] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:34.440955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac650d-43c3-4787-8b74-2fc8ec885e3b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:34.441407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac650d-43c3-4787-8b74-2fc8ec885e3b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:34.441503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac650d-43c3-4787-8b74-2fc8ec885e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:34.507381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cfac650d-43c3-4787-8b74-2fc8ec885e3b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:34.508636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cfac650d-43c3-4787-8b74-2fc8ec885e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:34.519275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:34.610845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 672/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:34.892660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517e8ed-09df-4025-815a-a5b92b4221a1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:34.894231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c517e8ed-09df-4025-815a-a5b92b4221a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:34.996571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517e8ed-09df-4025-815a-a5b92b4221a1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:35.068466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:35.069072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:35.069345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:35.069612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:35.071231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:35.107089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:35.180234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:35.181949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc77a31-e1a1-4e00-9d93-4a4386a1fd81 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:35.210235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 668/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:39.502752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9416cf-e80f-4e4e-8d2f-f99cbcf90e5f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:39.503353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9416cf-e80f-4e4e-8d2f-f99cbcf90e5f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:39.503353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9416cf-e80f-4e4e-8d2f-f99cbcf90e5f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:39.504706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 673/1341] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:39.510300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa8a2-1691-404c-8533-eb4ce79115ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:39.510771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa8a2-1691-404c-8533-eb4ce79115ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:39.510876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efa8a2-1691-404c-8533-eb4ce79115ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:39.580569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d4efa8a2-1691-404c-8533-eb4ce79115ad None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:39.581804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d4efa8a2-1691-404c-8533-eb4ce79115ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:39.673872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:39.685046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 669/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:39.976242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9416cf-e80f-4e4e-8d2f-f99cbcf90e5f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:39.977865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d9416cf-e80f-4e4e-8d2f-f99cbcf90e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:40.088007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9416cf-e80f-4e4e-8d2f-f99cbcf90e5f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:40.127421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:40.166006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:40.166552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:40.166829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:40.167096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:40.168724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:40.281417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:40.283019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b2e7a-1709-490c-b8c6-70aeb10cf6b3 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:40.302486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 674/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:43.275591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889bf3-5a21-4480-949d-f3719e4b3087 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:43.276124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889bf3-5a21-4480-949d-f3719e4b3087 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:43.276124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64889bf3-5a21-4480-949d-f3719e4b3087 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:43.277477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 670/1344] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:43.282949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cdb6c-ee45-47cd-9a7d-3074ea5ecb53 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:43.283377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cdb6c-ee45-47cd-9a7d-3074ea5ecb53 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:43.283491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cdb6c-ee45-47cd-9a7d-3074ea5ecb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:43.351275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ed0cdb6c-ee45-47cd-9a7d-3074ea5ecb53 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:43.352524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ed0cdb6c-ee45-47cd-9a7d-3074ea5ecb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:43.370923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:43.449920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 675/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:43.737726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889bf3-5a21-4480-949d-f3719e4b3087 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:43.739358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-64889bf3-5a21-4480-949d-f3719e4b3087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:43.851457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64889bf3-5a21-4480-949d-f3719e4b3087 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:43.929251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:43.929860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:43.930129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:43.930394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:43.932013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:43.986861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:44.055659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:44.057135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50389719-fb1e-4dc3-8cb4-7f61b93e9d61 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:44.075601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 671/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:48.652216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095ddd3-c3be-4ad9-a09d-acaae9e028dc None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:48.652776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095ddd3-c3be-4ad9-a09d-acaae9e028dc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:48.652776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095ddd3-c3be-4ad9-a09d-acaae9e028dc None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:48.654138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 676/1347] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:48.660581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5de5ae-5c91-43ee-8165-affe532bc4d7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:48.661259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5de5ae-5c91-43ee-8165-affe532bc4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:48.661259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5de5ae-5c91-43ee-8165-affe532bc4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:48.739633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5b5de5ae-5c91-43ee-8165-affe532bc4d7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:48.741004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5b5de5ae-5c91-43ee-8165-affe532bc4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:48.840894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:48.844181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 672/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:49.133520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095ddd3-c3be-4ad9-a09d-acaae9e028dc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:36:49.135039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4095ddd3-c3be-4ad9-a09d-acaae9e028dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:49.242425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095ddd3-c3be-4ad9-a09d-acaae9e028dc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:36:49.291926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:49.317586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:36:49.318197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:49.318476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:49.318749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:49.320361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:49.546763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:36:49.548202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e5635-24f7-4a76-9e31-40acf118d938 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:36:49.566553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 677/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:36:54.626639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68fe7a-c63d-41fd-abd4-c37db0013ef2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:54.627145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68fe7a-c63d-41fd-abd4-c37db0013ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:54.627700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68fe7a-c63d-41fd-abd4-c37db0013ef2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:54.631192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 673/1350] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:36:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:54.637413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e31f0d-f953-4437-94d5-8f69b7b1c476 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:54.637847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e31f0d-f953-4437-94d5-8f69b7b1c476 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:54.637957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e31f0d-f953-4437-94d5-8f69b7b1c476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:54.758037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f0e31f0d-f953-4437-94d5-8f69b7b1c476 None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:54.759406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0e31f0d-f953-4437-94d5-8f69b7b1c476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:54.786874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:54.901748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 678/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:36:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:36:59.274331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19421f-3039-4d68-81b8-0a77543386bf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:59.275325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19421f-3039-4d68-81b8-0a77543386bf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:59.275604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19421f-3039-4d68-81b8-0a77543386bf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:59.277162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 674/1352] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:36:59.282790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2eed4-caa6-42be-b5ab-d1d23e02f05e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:36:59.283239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2eed4-caa6-42be-b5ab-d1d23e02f05e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:36:59.283341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2eed4-caa6-42be-b5ab-d1d23e02f05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:36:59.366110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-04b2eed4-caa6-42be-b5ab-d1d23e02f05e None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:36:59.367479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-04b2eed4-caa6-42be-b5ab-d1d23e02f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:36:59.443984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:36:59.483964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 679/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:36:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:00.923986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5131ff3-9db7-4afb-88c8-1cb5546a87a2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:00.924517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5131ff3-9db7-4afb-88c8-1cb5546a87a2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:00.924610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5131ff3-9db7-4afb-88c8-1cb5546a87a2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:00.925902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 675/1354] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:00.931028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a79dd3-3d1a-4c71-b3c3-d320ec46f4bb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:00.931438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a79dd3-3d1a-4c71-b3c3-d320ec46f4bb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:00.931574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a79dd3-3d1a-4c71-b3c3-d320ec46f4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:01.008430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-94a79dd3-3d1a-4c71-b3c3-d320ec46f4bb None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:01.009732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94a79dd3-3d1a-4c71-b3c3-d320ec46f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:01.130380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 680/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:37:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:04.405244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7842e4-6bd4-49cb-a9e8-a5209c26639e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:04.405780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7842e4-6bd4-49cb-a9e8-a5209c26639e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:04.405780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7842e4-6bd4-49cb-a9e8-a5209c26639e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:04.407114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 676/1356] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:04.412517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7c53-cee6-4fa3-8147-1336fdd5c6b7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:04.412952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7c53-cee6-4fa3-8147-1336fdd5c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:04.413078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f7c53-cee6-4fa3-8147-1336fdd5c6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:04.449140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:04.484784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e76f7c53-cee6-4fa3-8147-1336fdd5c6b7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:04.486002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e76f7c53-cee6-4fa3-8147-1336fdd5c6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:04.628963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 681/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:04.929724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4093b2b-79b4-47ea-9c3f-c47dc6370c66 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:04.930006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4093b2b-79b4-47ea-9c3f-c47dc6370c66 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:04.930276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4093b2b-79b4-47ea-9c3f-c47dc6370c66 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:04.931619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 677/1358] 10.222.0.17 () {54 vars in 962 bytes} [Wed Aug 24 16:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:04.945051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c9fa-46ae-4827-81be-430bb4491018 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:04.947274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c9fa-46ae-4827-81be-430bb4491018 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:04.947274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c9fa-46ae-4827-81be-430bb4491018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:05.037843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cfc5c9fa-46ae-4827-81be-430bb4491018 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:05.039375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cfc5c9fa-46ae-4827-81be-430bb4491018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:05.090347 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:05.162740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 682/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Aug 24 16:37:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:05.172751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4093b2b-79b4-47ea-9c3f-c47dc6370c66 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:05.180938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e4093b2b-79b4-47ea-9c3f-c47dc6370c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:05.333053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4093b2b-79b4-47ea-9c3f-c47dc6370c66 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:05.412703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:05.454584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:05.455195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:05.455473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:05.455731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:05.457364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:05.554759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:05.556831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964687e4-11b8-49ce-a5e4-7d475cdbea40 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:05.578179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 678/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:07.720526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc268dd-333d-440d-80f0-054a4213c544 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:07.721089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc268dd-333d-440d-80f0-054a4213c544 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:07.721089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc268dd-333d-440d-80f0-054a4213c544 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:07.722468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 683/1361] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:07.727427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428cff7-05a2-4ef5-940d-a3e84fe84c18 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:07.727882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428cff7-05a2-4ef5-940d-a3e84fe84c18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:07.727991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428cff7-05a2-4ef5-940d-a3e84fe84c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:07.794204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b428cff7-05a2-4ef5-940d-a3e84fe84c18 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:07.795394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b428cff7-05a2-4ef5-940d-a3e84fe84c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:07.842468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:07.913274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 679/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:08.198498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc268dd-333d-440d-80f0-054a4213c544 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:08.200370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0cc268dd-333d-440d-80f0-054a4213c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:08.365282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc268dd-333d-440d-80f0-054a4213c544 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:08.375154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:08.487630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:08.488423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:08.488825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:08.489196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:08.490975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:08.589268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:08.591830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad45a8c-3ea9-4845-8834-a39952615439 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:08.611750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 684/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:11.069245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3906d-2167-470c-978f-3d5fef0a896d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:11.069764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3906d-2167-470c-978f-3d5fef0a896d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:11.069764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3906d-2167-470c-978f-3d5fef0a896d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:11.071123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 680/1364] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:11.076925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c35c4-1466-4e18-8d92-ddcc66fc3216 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:11.077388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c35c4-1466-4e18-8d92-ddcc66fc3216 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:11.077486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c35c4-1466-4e18-8d92-ddcc66fc3216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:11.094862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:11.164323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ad4c35c4-1466-4e18-8d92-ddcc66fc3216 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:11.165577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ad4c35c4-1466-4e18-8d92-ddcc66fc3216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:11.269491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 685/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:11.578946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559dfea-af02-4dec-b125-8c4f2ceeca39 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:11.579660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559dfea-af02-4dec-b125-8c4f2ceeca39 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:11.580061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559dfea-af02-4dec-b125-8c4f2ceeca39 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:11.581574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 681/1366] 10.222.0.17 () {54 vars in 962 bytes} [Wed Aug 24 16:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:11.587315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be5c4-add0-4fb3-8954-1d68e975db0e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:11.587752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be5c4-add0-4fb3-8954-1d68e975db0e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:11.588028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be5c4-add0-4fb3-8954-1d68e975db0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:11.672282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5f8be5c4-add0-4fb3-8954-1d68e975db0e None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:11.673451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5f8be5c4-add0-4fb3-8954-1d68e975db0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:11.711476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:11.801353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 686/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Aug 24 16:37:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:11.810162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559dfea-af02-4dec-b125-8c4f2ceeca39 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:11.811853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6559dfea-af02-4dec-b125-8c4f2ceeca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:12.004550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559dfea-af02-4dec-b125-8c4f2ceeca39 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:12.029546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:12.131568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:12.132243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:12.132546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:12.132753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:12.134451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:12.248521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:12.250531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb10ca-ddf3-4c99-863f-91dd11326bdc service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:12.276377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 682/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:12.801270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f289f-c7cf-491c-8b33-d31a0af949d8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:12.801833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f289f-c7cf-491c-8b33-d31a0af949d8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:12.802306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f289f-c7cf-491c-8b33-d31a0af949d8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:12.803798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 687/1369] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:12.809226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564f96-4624-4edd-ba37-c3a484388b4d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:12.809501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564f96-4624-4edd-ba37-c3a484388b4d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:12.809792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564f96-4624-4edd-ba37-c3a484388b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:12.843794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:12.879652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2564f96-4624-4edd-ba37-c3a484388b4d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:12.881727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2564f96-4624-4edd-ba37-c3a484388b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:12.992492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 683/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:13.605601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a639c-0a23-484c-b6de-d9c3e18021a8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:13.606784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a639c-0a23-484c-b6de-d9c3e18021a8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:13.607367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a639c-0a23-484c-b6de-d9c3e18021a8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:13.609384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 688/1371] 10.222.0.17 () {54 vars in 964 bytes} [Wed Aug 24 16:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:13.614114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac795-8d2b-494e-98fc-a874323cc44a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:13.614521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac795-8d2b-494e-98fc-a874323cc44a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:13.614729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac795-8d2b-494e-98fc-a874323cc44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:13.717492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-135ac795-8d2b-494e-98fc-a874323cc44a None None] MFA Rules not processed for user `ef06aba40382479289b1e9e19395a2be`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:13.719128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-135ac795-8d2b-494e-98fc-a874323cc44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:13.773045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:13.812750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 684/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Aug 24 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:13.820569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a639c-0a23-484c-b6de-d9c3e18021a8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:13.822683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a10a639c-0a23-484c-b6de-d9c3e18021a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:13.977002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a639c-0a23-484c-b6de-d9c3e18021a8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:14.023347 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:14.061400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:14.062235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:14.062628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:14.063012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:14.065586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:14.209609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:14.211344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c22f1-9f35-4d61-a4d8-10f2514a766d service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:14.237729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 689/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:14.437385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed950bb2-ffe6-4397-b478-a2aebe100668 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:14.437599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed950bb2-ffe6-4397-b478-a2aebe100668 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:14.437847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed950bb2-ffe6-4397-b478-a2aebe100668 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:14.439191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 685/1374] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:14.444396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c0703-0ff7-4dc6-ac13-54ea8ce515d3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:14.444687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c0703-0ff7-4dc6-ac13-54ea8ce515d3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:14.445172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c0703-0ff7-4dc6-ac13-54ea8ce515d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:14.512030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d64c0703-0ff7-4dc6-ac13-54ea8ce515d3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:14.513261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d64c0703-0ff7-4dc6-ac13-54ea8ce515d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:14.533830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:14.611713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 690/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:14.903136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed950bb2-ffe6-4397-b478-a2aebe100668 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:14.904685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed950bb2-ffe6-4397-b478-a2aebe100668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:15.052880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed950bb2-ffe6-4397-b478-a2aebe100668 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:15.103386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:15.169665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:15.170199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:15.170477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:15.170745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:15.172393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:15.272968 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:15.274758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30816a0c-3c0c-423f-a7b7-1183b7a0a0e8 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:15.294384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 686/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:17.368214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20fd7-13ae-454b-b607-8335f8925c79 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:17.368709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20fd7-13ae-454b-b607-8335f8925c79 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:17.368936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20fd7-13ae-454b-b607-8335f8925c79 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:17.370302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 691/1377] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:17.376929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e219b-3b1e-4d21-8e37-63bdafce215c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:17.377373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e219b-3b1e-4d21-8e37-63bdafce215c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:17.377482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e219b-3b1e-4d21-8e37-63bdafce215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:17.446382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-749e219b-3b1e-4d21-8e37-63bdafce215c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:17.447501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-749e219b-3b1e-4d21-8e37-63bdafce215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:17.454239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:17.560382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 687/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:17.858318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20fd7-13ae-454b-b607-8335f8925c79 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:17.860128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-11d20fd7-13ae-454b-b607-8335f8925c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:18.079243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20fd7-13ae-454b-b607-8335f8925c79 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:18.086198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:18.227618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:18.228790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:18.229310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:18.229801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:18.233849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:18.326104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:18.327684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88266e-c348-49b5-b001-ef6bcd5d35d3 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:18.348258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 692/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:20.449981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becfd43-4051-469d-8f79-29d5cd70f4af None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:20.450535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becfd43-4051-469d-8f79-29d5cd70f4af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:20.450535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becfd43-4051-469d-8f79-29d5cd70f4af None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:20.451992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 688/1380] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:20.457115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f5824-84b3-4bbb-ae92-1ebf592e6499 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:20.457527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f5824-84b3-4bbb-ae92-1ebf592e6499 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:20.457633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f5824-84b3-4bbb-ae92-1ebf592e6499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:20.469530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:20.534247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8f5f5824-84b3-4bbb-ae92-1ebf592e6499 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:20.536125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8f5f5824-84b3-4bbb-ae92-1ebf592e6499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:20.650122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 693/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:37:20.946724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becfd43-4051-469d-8f79-29d5cd70f4af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:20.948459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0becfd43-4051-469d-8f79-29d5cd70f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:21.115987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:21.123879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becfd43-4051-469d-8f79-29d5cd70f4af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:21.246392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:21.247149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:21.247524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:21.247893 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:21.249633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:21.350831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:21.352666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f63eeb-98d2-498e-8221-7d1245c56d8b service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:21.364219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:21.373141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 689/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:23.492249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95dde-cb10-467f-ba75-7c1d093ded78 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:23.493190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95dde-cb10-467f-ba75-7c1d093ded78 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:23.493190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95dde-cb10-467f-ba75-7c1d093ded78 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:23.494191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 694/1383] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:23.501216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a97f2-412a-4c61-9291-ed6317867447 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:23.501754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a97f2-412a-4c61-9291-ed6317867447 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:23.502442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a97f2-412a-4c61-9291-ed6317867447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:23.623443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f88a97f2-412a-4c61-9291-ed6317867447 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:23.625684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f88a97f2-412a-4c61-9291-ed6317867447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:23.732559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 690/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:24.027469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f95dde-cb10-467f-ba75-7c1d093ded78 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:24.029455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-32f95dde-cb10-467f-ba75-7c1d093ded78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:24.129618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:24.197449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f95dde-cb10-467f-ba75-7c1d093ded78 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:24.325972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:24.326862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:24.327296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:24.327763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:24.329419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:24.392874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:24.459756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:24.462838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270006-2a30-47d1-a47d-6f1cd519728a service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:24.491590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 695/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:37:26.676017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c09e75-cbd8-47a7-9d4a-d0bf9abb1622 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:26.676703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c09e75-cbd8-47a7-9d4a-d0bf9abb1622 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:26.676703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c09e75-cbd8-47a7-9d4a-d0bf9abb1622 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:26.677937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 691/1386] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:26.685228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d24240-ce00-4fef-a348-eefcddf3c9dc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:26.689480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d24240-ce00-4fef-a348-eefcddf3c9dc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:26.689480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d24240-ce00-4fef-a348-eefcddf3c9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:26.765746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e3d24240-ce00-4fef-a348-eefcddf3c9dc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:26.766966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e3d24240-ce00-4fef-a348-eefcddf3c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:26.875480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 696/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:27.242083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c09e75-cbd8-47a7-9d4a-d0bf9abb1622 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:27.243616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a6c09e75-cbd8-47a7-9d4a-d0bf9abb1622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:27.326485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:27.405340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c09e75-cbd8-47a7-9d4a-d0bf9abb1622 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:27.547794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:27.548622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:27.548926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:27.549261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:27.551171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:27.593219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:27.662587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:27.665787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10e153-6138-4acb-884d-1155e2d8eec6 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:27.701634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 692/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:29.725260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347aea2-9d3f-48f1-8e67-005622650525 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:29.725800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347aea2-9d3f-48f1-8e67-005622650525 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:29.725800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347aea2-9d3f-48f1-8e67-005622650525 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:29.727173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 697/1389] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:29.733395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150d061-cb4e-405b-9b64-8ee589b10fe5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:29.733844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150d061-cb4e-405b-9b64-8ee589b10fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:29.734083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150d061-cb4e-405b-9b64-8ee589b10fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:29.774911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4316-a7da-4139-9f3c-1011ad558132 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:29.775190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4316-a7da-4139-9f3c-1011ad558132 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:29.775460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c4316-a7da-4139-9f3c-1011ad558132 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:29.776883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 698/1390] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:37:29.781922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408283-1d39-40db-ba87-41b19e13c99a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:29.782203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408283-1d39-40db-ba87-41b19e13c99a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:29.782476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0408283-1d39-40db-ba87-41b19e13c99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:29.801457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7150d061-cb4e-405b-9b64-8ee589b10fe5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:29.802687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7150d061-cb4e-405b-9b64-8ee589b10fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:29.820110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:29.854356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c0408283-1d39-40db-ba87-41b19e13c99a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:29.856206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c0408283-1d39-40db-ba87-41b19e13c99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:29.906680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 693/1391] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:29.954950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 699/1392] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:37:30.218423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e4504-e00a-4d34-9188-d8d824fd4ea8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:30.218714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e4504-e00a-4d34-9188-d8d824fd4ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:30.218987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e4504-e00a-4d34-9188-d8d824fd4ea8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:30.220334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 694/1393] 10.222.0.17 () {54 vars in 967 bytes} [Wed Aug 24 16:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:30.225327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:30.226473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:30.226818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:30.246619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e4504-e00a-4d34-9188-d8d824fd4ea8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:30.248371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-888e4504-e00a-4d34-9188-d8d824fd4ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:30.262560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:30.307121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] MFA Rules not processed for user `43cdc8de558d4087bed8a3bb48cf4bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:30.309088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:30.360372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:30.390483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 700/1394] 10.222.0.17 () {58 vars in 1064 bytes} [Wed Aug 24 16:37:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:30.399067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:30.400013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e4504-e00a-4d34-9188-d8d824fd4ea8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:30.407549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d97be-6611-4879-a3de-2879dca484c7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:30.483881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:30.484531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:30.484812 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:30.488877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:30.488877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:30.490231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:30.518017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b1756-c376-433a-a03f-645261891c4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:30.518630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b1756-c376-433a-a03f-645261891c4c service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:30.518916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b1756-c376-433a-a03f-645261891c4c service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:30.519183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b1756-c376-433a-a03f-645261891c4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:30.521062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-402b1756-c376-433a-a03f-645261891c4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:30.605915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:30.607368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524d16f-7038-4d80-bff7-fec05792242e service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:30.613764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b1756-c376-433a-a03f-645261891c4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:30.615409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b1756-c376-433a-a03f-645261891c4c service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:30.622459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:30.627031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 701/1395] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:37:30.634709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 695/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:31.502576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02747a3a-5240-4d86-8dbd-07c94dad4d72 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:31.503046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02747a3a-5240-4d86-8dbd-07c94dad4d72 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:31.503136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02747a3a-5240-4d86-8dbd-07c94dad4d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:31.575040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-02747a3a-5240-4d86-8dbd-07c94dad4d72 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:31.576827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-02747a3a-5240-4d86-8dbd-07c94dad4d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:31.674676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 702/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:37:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:31.719101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb95db-bfe3-4d54-ab54-8af5cf9f323e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:31.719593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb95db-bfe3-4d54-ab54-8af5cf9f323e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:31.719690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb95db-bfe3-4d54-ab54-8af5cf9f323e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:31.720992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 696/1398] 10.222.0.17 () {54 vars in 943 bytes} [Wed Aug 24 16:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:31.727048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13f83d-0ea1-481e-9511-d97048ad958d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:31.727342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13f83d-0ea1-481e-9511-d97048ad958d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:31.727643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13f83d-0ea1-481e-9511-d97048ad958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:31.810119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9c13f83d-0ea1-481e-9511-d97048ad958d None None] MFA Rules not processed for user `1f762db26df84f5faf7e6778383e2870`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:31.811345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9c13f83d-0ea1-481e-9511-d97048ad958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:31.844109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:31.927299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 703/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Wed Aug 24 16:37:31 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:31.937940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb95db-bfe3-4d54-ab54-8af5cf9f323e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:31.939550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b1fb95db-bfe3-4d54-ab54-8af5cf9f323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:32.048921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:32.092188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb95db-bfe3-4d54-ab54-8af5cf9f323e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:32.211647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:32.212540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:32.212825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:32.213096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:32.215401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:32.309383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:32.311170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867d74c-3a6c-4648-a831-6d894234a6d2 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:32.314758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:32.320178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 697/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:37:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:37:32.791341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ddc7d-8cd4-4aef-8b58-1e53abb6922d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:32.791807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ddc7d-8cd4-4aef-8b58-1e53abb6922d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:32.792692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ddc7d-8cd4-4aef-8b58-1e53abb6922d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:32.794040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 704/1401] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:32.799816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57016daa-a485-4135-b48b-d2b09a14ec43 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:32.800302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57016daa-a485-4135-b48b-d2b09a14ec43 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:32.800448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57016daa-a485-4135-b48b-d2b09a14ec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:32.872811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-57016daa-a485-4135-b48b-d2b09a14ec43 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:32.874102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57016daa-a485-4135-b48b-d2b09a14ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:32.977497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 698/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:33.267647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ddc7d-8cd4-4aef-8b58-1e53abb6922d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:33.269244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ab8ddc7d-8cd4-4aef-8b58-1e53abb6922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:33.439850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ddc7d-8cd4-4aef-8b58-1e53abb6922d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:33.443984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:33.573885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:33.574416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:33.574670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:33.574943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:33.576568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:33.674239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:33.675917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369fc47-3459-483f-8edf-2a50b26e14fa service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:33.694497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 705/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:35.771045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f3e2c-faae-45ff-aef3-9cc171332188 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:35.771693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f3e2c-faae-45ff-aef3-9cc171332188 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:35.771693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f3e2c-faae-45ff-aef3-9cc171332188 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:35.772942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 699/1404] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:35.778396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a3859-1bf1-48c7-a3da-cfffaf6d74c8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:35.778871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a3859-1bf1-48c7-a3da-cfffaf6d74c8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:35.778967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a3859-1bf1-48c7-a3da-cfffaf6d74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:35.788665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:35.847405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9b0a3859-1bf1-48c7-a3da-cfffaf6d74c8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:35.848653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9b0a3859-1bf1-48c7-a3da-cfffaf6d74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:35.960238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 706/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:37:36.262073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f3e2c-faae-45ff-aef3-9cc171332188 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:36.263583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-224f3e2c-faae-45ff-aef3-9cc171332188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:36.351351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:36.427511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f3e2c-faae-45ff-aef3-9cc171332188 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:36.555510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:36.556040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:36.556318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:36.559824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:36.559824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:36.607273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:36.667892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:36.669853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0042bf4-6831-4904-bb45-2bc3e6523ceb service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:36.690836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 700/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:38.836773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029984-6ed7-42aa-95d0-8339ff54b632 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:38.837346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029984-6ed7-42aa-95d0-8339ff54b632 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:38.837346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029984-6ed7-42aa-95d0-8339ff54b632 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:38.839234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 707/1407] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:38.844743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769822dd-dd2f-45e1-ac63-ce610d147253 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:38.845699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769822dd-dd2f-45e1-ac63-ce610d147253 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:38.846037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769822dd-dd2f-45e1-ac63-ce610d147253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:38.920142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-769822dd-dd2f-45e1-ac63-ce610d147253 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:38.921613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-769822dd-dd2f-45e1-ac63-ce610d147253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:39.029930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 701/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:39.335032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029984-6ed7-42aa-95d0-8339ff54b632 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:39.336637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-39029984-6ed7-42aa-95d0-8339ff54b632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:39.434056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:39.508031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029984-6ed7-42aa-95d0-8339ff54b632 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:39.636583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:39.637114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:39.637390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:39.637664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:39.639274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:39.709777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:39.781950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:39.783641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b147618-4aa9-475c-ba91-dbdf16261dcd service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:39.811501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 708/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:37:49.027757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:49.809097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=24 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:56.307081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef8555-f47c-4ce0-a253-9b0be99a55cf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:56.307663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef8555-f47c-4ce0-a253-9b0be99a55cf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:56.307663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef8555-f47c-4ce0-a253-9b0be99a55cf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:56.309111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 702/1410] 10.222.0.17 () {54 vars in 918 bytes} [Wed Aug 24 16:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:56.314273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab7a44-b4b8-4f26-ac72-53ae602d7507 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:56.314688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab7a44-b4b8-4f26-ac72-53ae602d7507 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:56.314799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab7a44-b4b8-4f26-ac72-53ae602d7507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:56.392187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f0ab7a44-b4b8-4f26-ac72-53ae602d7507 None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:56.393425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0ab7a44-b4b8-4f26-ac72-53ae602d7507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:56.498051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 709/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 24 16:37:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:56.512440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2faf8e-9faf-430a-9b40-e97911c9f155 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:56.512719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2faf8e-9faf-430a-9b40-e97911c9f155 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:56.512985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2faf8e-9faf-430a-9b40-e97911c9f155 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:56.514358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 703/1412] 10.222.0.17 () {54 vars in 918 bytes} [Wed Aug 24 16:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:56.518946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b409ceb-a033-44a8-b6e6-7d0b77d616e5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:56.519229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b409ceb-a033-44a8-b6e6-7d0b77d616e5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:56.519499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b409ceb-a033-44a8-b6e6-7d0b77d616e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:56.592187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:56.594623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9b409ceb-a033-44a8-b6e6-7d0b77d616e5 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:56.595821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9b409ceb-a033-44a8-b6e6-7d0b77d616e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:56.713617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 710/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Aug 24 16:37:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:59.117989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4505fd-0d6a-4b20-b3a1-57928c6d5b33 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:59.118522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4505fd-0d6a-4b20-b3a1-57928c6d5b33 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:59.118522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4505fd-0d6a-4b20-b3a1-57928c6d5b33 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:59.119838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:37:59.125354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687ad53-1427-4985-b812-94dd872a6bad None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:59.125780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687ad53-1427-4985-b812-94dd872a6bad None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:59.125912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687ad53-1427-4985-b812-94dd872a6bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:59.191265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9687ad53-1427-4985-b812-94dd872a6bad None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:37:59.192520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9687ad53-1427-4985-b812-94dd872a6bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:59.248157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:59.291684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:37:59.579751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4505fd-0d6a-4b20-b3a1-57928c6d5b33 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:37:59.581332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7c4505fd-0d6a-4b20-b3a1-57928c6d5b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:59.729706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4505fd-0d6a-4b20-b3a1-57928c6d5b33 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:37:59.801857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:37:59.841582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:37:59.842110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:37:59.842383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:37:59.842652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:37:59.844222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:37:59.932576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:37:59.934188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eab02f-002b-4224-8f76-5ec499254332 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:37:59.951703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 705/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:00.850068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04437c97-5281-42cb-93d9-b71ca1a6160b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:00.850608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04437c97-5281-42cb-93d9-b71ca1a6160b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:00.850608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04437c97-5281-42cb-93d9-b71ca1a6160b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:00.851947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:00.857326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade5b1-6acd-4899-9bec-abdab63dbb72 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:00.857800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade5b1-6acd-4899-9bec-abdab63dbb72 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:00.857899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade5b1-6acd-4899-9bec-abdab63dbb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:00.935692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-84ade5b1-6acd-4899-9bec-abdab63dbb72 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:00.936929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-84ade5b1-6acd-4899-9bec-abdab63dbb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:00.963543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:01.034333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:01.185874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04437c97-5281-42cb-93d9-b71ca1a6160b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:01.187365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-04437c97-5281-42cb-93d9-b71ca1a6160b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:01.334981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04437c97-5281-42cb-93d9-b71ca1a6160b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:01.375175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:01.447770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:01.448284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:01.448589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:01.448856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:01.450445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:01.542727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:01.544328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575356e-0893-4a0b-bcdf-fa747ffd3b0a service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:01.562625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 713/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:02.524871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfc4e4-da85-44a8-9c6a-16c62543fc88 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:02.525426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfc4e4-da85-44a8-9c6a-16c62543fc88 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:02.525524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bfc4e4-da85-44a8-9c6a-16c62543fc88 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:02.526821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:02.531840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e59e2-cac4-4da9-9c8a-42198ad98997 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:02.532253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e59e2-cac4-4da9-9c8a-42198ad98997 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:02.532405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e59e2-cac4-4da9-9c8a-42198ad98997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:02.587482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:02.608951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dd9e59e2-cac4-4da9-9c8a-42198ad98997 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:02.610172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dd9e59e2-cac4-4da9-9c8a-42198ad98997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:02.710897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:02.868741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfc4e4-da85-44a8-9c6a-16c62543fc88 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:02.870356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d5bfc4e4-da85-44a8-9c6a-16c62543fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:03.018134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bfc4e4-da85-44a8-9c6a-16c62543fc88 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:03.032489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:03.137780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:03.138294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:03.138570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:03.138839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:03.140464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:03.231292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:03.232957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa613f0-da21-40fa-a285-531d72b1569a service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:03.250997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 708/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:04.114825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84e561-f310-4ceb-a45c-a9218e37e63f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:04.115391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84e561-f310-4ceb-a45c-a9218e37e63f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:04.115391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84e561-f310-4ceb-a45c-a9218e37e63f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:04.116756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:04.121917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeae38-0bf6-430f-8cca-073e30c8d235 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:04.122345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeae38-0bf6-430f-8cca-073e30c8d235 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:04.122440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeae38-0bf6-430f-8cca-073e30c8d235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:04.143527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:04.200776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8dfeae38-0bf6-430f-8cca-073e30c8d235 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:04.201892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8dfeae38-0bf6-430f-8cca-073e30c8d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:04.296687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:04.447560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84e561-f310-4ceb-a45c-a9218e37e63f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:04.449128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6e84e561-f310-4ceb-a45c-a9218e37e63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:04.560393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:04.597285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84e561-f310-4ceb-a45c-a9218e37e63f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:04.708744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:04.709341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:04.709616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:04.709883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:04.712036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:04.789074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:04.803576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:04.805944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d0b06-1000-4039-acb4-a3a6349ecf41 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:04.824225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 716/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:05.697777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f8c1-3fa8-4758-9952-4d55af66b80d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:05.698343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f8c1-3fa8-4758-9952-4d55af66b80d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:05.698343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f8c1-3fa8-4758-9952-4d55af66b80d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:05.699834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:38:05.704805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ce10a-29a9-471b-ae23-5760ad6b9edf None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:05.705201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ce10a-29a9-471b-ae23-5760ad6b9edf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:05.705349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ce10a-29a9-471b-ae23-5760ad6b9edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:05.783395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cc2ce10a-29a9-471b-ae23-5760ad6b9edf None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:05.784634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cc2ce10a-29a9-471b-ae23-5760ad6b9edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:05.880582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:06.033698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f8c1-3fa8-4758-9952-4d55af66b80d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:06.035208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4266f8c1-3fa8-4758-9952-4d55af66b80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:06.119059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:06.183300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f8c1-3fa8-4758-9952-4d55af66b80d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:06.296538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:06.297142 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:06.297421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:06.297693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:06.299347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:06.349917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:06.392611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:06.394282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81538743-ff95-4313-b2b8-4b66b85648b5 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:06.416800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 711/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:07.280893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23101e20-c382-4800-8d16-62782d7222d8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:07.281454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23101e20-c382-4800-8d16-62782d7222d8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:07.281454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23101e20-c382-4800-8d16-62782d7222d8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:07.282767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:07.287611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c90fef-7688-42ea-a283-ce13be6a92a2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:07.288039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c90fef-7688-42ea-a283-ce13be6a92a2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:07.288172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c90fef-7688-42ea-a283-ce13be6a92a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:07.366234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-35c90fef-7688-42ea-a283-ce13be6a92a2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:07.368067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-35c90fef-7688-42ea-a283-ce13be6a92a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:07.469342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:07.624282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23101e20-c382-4800-8d16-62782d7222d8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:07.625819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-23101e20-c382-4800-8d16-62782d7222d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:07.674938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:07.771621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23101e20-c382-4800-8d16-62782d7222d8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:07.883306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:07.883835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:07.884114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:07.884409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:07.886015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:07.893167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:07.977326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:07.978918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7e3a0-0b12-47ae-91c7-bd9c5b1d2989 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:07.997084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 719/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:08.878128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1aadb-b9cc-49d6-a6d2-bc592c0dccbf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:08.878699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1aadb-b9cc-49d6-a6d2-bc592c0dccbf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:08.878699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1aadb-b9cc-49d6-a6d2-bc592c0dccbf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:08.879991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:08.885569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb34ba-3079-413a-9318-54d8c21e626f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:08.886026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb34ba-3079-413a-9318-54d8c21e626f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:08.886131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb34ba-3079-413a-9318-54d8c21e626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:08.960519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7acb34ba-3079-413a-9318-54d8c21e626f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:08.961828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7acb34ba-3079-413a-9318-54d8c21e626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:09.028023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:09.057653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:09.209408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1aadb-b9cc-49d6-a6d2-bc592c0dccbf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:09.210863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21f1aadb-b9cc-49d6-a6d2-bc592c0dccbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:09.229607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:09.358315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1aadb-b9cc-49d6-a6d2-bc592c0dccbf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:09.446849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:09.473020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:09.473561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:09.473838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:09.474106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:09.475737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:09.568934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:09.571098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c41c82-fadb-4bec-848f-497dfa884b53 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:09.589500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 714/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:10.476183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3951a02-861c-4654-8d5b-e72b59f601fb None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:10.476823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3951a02-861c-4654-8d5b-e72b59f601fb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:10.476823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3951a02-861c-4654-8d5b-e72b59f601fb None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:10.480326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:10.485452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e73b5e-634a-4658-af16-e97e8c6d15fc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:10.491117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e73b5e-634a-4658-af16-e97e8c6d15fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:10.491117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e73b5e-634a-4658-af16-e97e8c6d15fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:10.562933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a6e73b5e-634a-4658-af16-e97e8c6d15fc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:10.564124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a6e73b5e-634a-4658-af16-e97e8c6d15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:10.607095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:10.660881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:10.812064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3951a02-861c-4654-8d5b-e72b59f601fb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:10.813593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e3951a02-861c-4654-8d5b-e72b59f601fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:10.959003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3951a02-861c-4654-8d5b-e72b59f601fb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:11.013089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:11.076116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:11.076749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:11.077026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:11.077296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:11.078888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:11.169586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:11.171122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac85d2-57eb-437c-a17c-81153090d82a service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:11.188719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 722/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:12.029121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4641-aa82-44e1-910a-6a1a1d4da2ba None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:12.029686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4641-aa82-44e1-910a-6a1a1d4da2ba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:12.029686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4641-aa82-44e1-910a-6a1a1d4da2ba None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:12.031002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:12.035789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111126b-6917-4c85-897a-f5b79ba8f1cc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:12.036211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111126b-6917-4c85-897a-f5b79ba8f1cc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:12.036316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111126b-6917-4c85-897a-f5b79ba8f1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:12.112232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8111126b-6917-4c85-897a-f5b79ba8f1cc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:12.113464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8111126b-6917-4c85-897a-f5b79ba8f1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:12.115443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:12.209165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:12.360925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c4641-aa82-44e1-910a-6a1a1d4da2ba None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:12.362448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e3c4641-aa82-44e1-910a-6a1a1d4da2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:12.514067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c4641-aa82-44e1-910a-6a1a1d4da2ba None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:12.541633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:12.628001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:12.628629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:12.628904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:12.629168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:12.630763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:12.722603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:12.725307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0a29c-76ee-42bf-84f4-017e60b39e5c service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:12.745248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 717/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:14.588029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a3ad4-c6e3-4772-a122-7c2a64e0c48e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:14.588628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a3ad4-c6e3-4772-a122-7c2a64e0c48e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:14.588628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a3ad4-c6e3-4772-a122-7c2a64e0c48e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:14.589947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:14.595187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe38eb-59af-49d8-908d-951cef71fae0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:14.595647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe38eb-59af-49d8-908d-951cef71fae0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:14.595755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe38eb-59af-49d8-908d-951cef71fae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:14.637686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:14.662841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4dbe38eb-59af-49d8-908d-951cef71fae0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:14.664068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4dbe38eb-59af-49d8-908d-951cef71fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:14.760112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:15.053267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a3ad4-c6e3-4772-a122-7c2a64e0c48e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:15.054808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ab6a3ad4-c6e3-4772-a122-7c2a64e0c48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:15.158729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a3ad4-c6e3-4772-a122-7c2a64e0c48e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:15.191008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:15.233925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:15.234528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:15.234805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:15.235072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:15.236704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:15.347211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:15.348748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd87acb-88b0-4163-94a9-0ebd4021a4ac service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:15.366981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 725/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:17.422838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc923a-4bde-4fb9-be53-e65cdff03719 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:17.423542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc923a-4bde-4fb9-be53-e65cdff03719 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:17.423542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc923a-4bde-4fb9-be53-e65cdff03719 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:17.424842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:17.430267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5004bf-8689-4752-802a-b18cebd95d23 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:17.430820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5004bf-8689-4752-802a-b18cebd95d23 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:17.430928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5004bf-8689-4752-802a-b18cebd95d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:17.498393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5f5004bf-8689-4752-802a-b18cebd95d23 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:17.499646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5f5004bf-8689-4752-802a-b18cebd95d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:17.512995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:17.596558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:17.889366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc923a-4bde-4fb9-be53-e65cdff03719 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:17.890902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-31cc923a-4bde-4fb9-be53-e65cdff03719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:17.994231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc923a-4bde-4fb9-be53-e65cdff03719 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:18.006484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:18.068992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:18.069589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:18.069861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:18.070122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:18.072615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:18.189770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:18.191122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020bcfc-89a6-404c-bf55-c2744794f71c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:18.209786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 720/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:20.552856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f5a83-b0b5-4e06-ac81-77a657e87ae0 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:20.554544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f5a83-b0b5-4e06-ac81-77a657e87ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:20.554544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f5a83-b0b5-4e06-ac81-77a657e87ae0 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:20.554854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:20.560430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5003-2cd6-44b2-86f9-c139e778c25e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:20.560873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5003-2cd6-44b2-86f9-c139e778c25e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:20.560995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf5003-2cd6-44b2-86f9-c139e778c25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:20.611257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:20.629601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9daf5003-2cd6-44b2-86f9-c139e778c25e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:20.630827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9daf5003-2cd6-44b2-86f9-c139e778c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:20.730552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:21.029540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f5a83-b0b5-4e06-ac81-77a657e87ae0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:21.031060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9d0f5a83-b0b5-4e06-ac81-77a657e87ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:21.132444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f5a83-b0b5-4e06-ac81-77a657e87ae0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:21.173957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:21.208639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:21.209431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:21.209933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:21.210526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:21.213104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:21.322078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:21.323509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866f421-4370-4dad-9aee-b8576e134e4c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:21.343364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 728/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:23.379655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42421-99ab-4e5f-9fbe-8faced457981 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:23.380222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42421-99ab-4e5f-9fbe-8faced457981 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:23.380222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b42421-99ab-4e5f-9fbe-8faced457981 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:23.381649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:23.387089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a06f-174f-47d9-b210-c44b3f235114 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:23.387528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a06f-174f-47d9-b210-c44b3f235114 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:23.387624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a06f-174f-47d9-b210-c44b3f235114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:23.453005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f284a06f-174f-47d9-b210-c44b3f235114 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:23.454205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f284a06f-174f-47d9-b210-c44b3f235114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:23.474676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:23.551217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:23.835962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42421-99ab-4e5f-9fbe-8faced457981 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:23.838415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-00b42421-99ab-4e5f-9fbe-8faced457981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:23.950172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b42421-99ab-4e5f-9fbe-8faced457981 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:23.972065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:24.027187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:24.027799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:24.028079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:24.028368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:24.030020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:24.137380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:24.138762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4deeb-ee77-495b-9496-6eec78767ef6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:24.157268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 723/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:26.190438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09d853-f403-4d4d-bd53-c8bd0f0fd62b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:26.190944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09d853-f403-4d4d-bd53-c8bd0f0fd62b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:26.190944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09d853-f403-4d4d-bd53-c8bd0f0fd62b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:26.192268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:26.197791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a8b40-bcb5-4991-9000-33081c993f53 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:26.198184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a8b40-bcb5-4991-9000-33081c993f53 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:26.198356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a8b40-bcb5-4991-9000-33081c993f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:26.263127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:26.265818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ef3a8b40-bcb5-4991-9000-33081c993f53 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:26.267062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ef3a8b40-bcb5-4991-9000-33081c993f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:26.366268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:26.660673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09d853-f403-4d4d-bd53-c8bd0f0fd62b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:26.662220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3a09d853-f403-4d4d-bd53-c8bd0f0fd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:26.766394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09d853-f403-4d4d-bd53-c8bd0f0fd62b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:26.815870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:26.840807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:26.841411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:26.841687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:26.841955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:26.844367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:26.953513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:26.954952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976787b2-6079-4399-9295-d9ac4269211d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:26.973977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 731/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:28.986364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b9334-be19-4378-a219-f6ae0f5c23e3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:28.986851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b9334-be19-4378-a219-f6ae0f5c23e3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:28.986946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b9334-be19-4378-a219-f6ae0f5c23e3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:28.988255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:28.994379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2e9ba-f342-4810-b876-bba064703c2b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:28.994897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2e9ba-f342-4810-b876-bba064703c2b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:28.994996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2e9ba-f342-4810-b876-bba064703c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:29.060947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-06f2e9ba-f342-4810-b876-bba064703c2b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:29.062141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-06f2e9ba-f342-4810-b876-bba064703c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:29.091142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:29.160418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:38:29.444107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b9334-be19-4378-a219-f6ae0f5c23e3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:29.446310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d51b9334-be19-4378-a219-f6ae0f5c23e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:29.550157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b9334-be19-4378-a219-f6ae0f5c23e3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:29.578534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:29.623839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:29.624484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:29.624761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:29.625028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:29.626618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:29.734345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:29.735706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce7465-a81b-43fe-99f0-060e148dbfd4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:29.753883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 726/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:32.304822 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df0620-e9e5-4d8e-b8fb-a8f3977e1729 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:32.305388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df0620-e9e5-4d8e-b8fb-a8f3977e1729 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:32.305388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df0620-e9e5-4d8e-b8fb-a8f3977e1729 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:32.306731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:32.312073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13f56f-2ade-4da5-b2ea-86d27a0641df None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:32.312499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13f56f-2ade-4da5-b2ea-86d27a0641df None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:32.312661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13f56f-2ade-4da5-b2ea-86d27a0641df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:32.378001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7d13f56f-2ade-4da5-b2ea-86d27a0641df None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:32.379193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7d13f56f-2ade-4da5-b2ea-86d27a0641df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:32.388076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:32.479796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:32.764901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df0620-e9e5-4d8e-b8fb-a8f3977e1729 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:32.766445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b0df0620-e9e5-4d8e-b8fb-a8f3977e1729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:32.869853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df0620-e9e5-4d8e-b8fb-a8f3977e1729 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:32.927272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:32.945929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:32.946529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:32.946786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:32.947055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:32.948663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:33.058054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:33.059397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9b5ba-6d1a-419f-b531-02ec62139a1a service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:33.077727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 734/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:35.072119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3a25-fa3f-40bc-ba2e-897c6d98a206 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:35.072682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3a25-fa3f-40bc-ba2e-897c6d98a206 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:35.072682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a3a25-fa3f-40bc-ba2e-897c6d98a206 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:35.074082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:35.079696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07552cd1-2ea1-4d26-aea7-efa827b82a62 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:35.080089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07552cd1-2ea1-4d26-aea7-efa827b82a62 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:35.080230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07552cd1-2ea1-4d26-aea7-efa827b82a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:35.146664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-07552cd1-2ea1-4d26-aea7-efa827b82a62 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:35.154425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-07552cd1-2ea1-4d26-aea7-efa827b82a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:35.187316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:35.245147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:35.533971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a3a25-fa3f-40bc-ba2e-897c6d98a206 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:35.535504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-859a3a25-fa3f-40bc-ba2e-897c6d98a206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:35.651857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a3a25-fa3f-40bc-ba2e-897c6d98a206 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:35.700527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:35.740529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:35.741136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:35.741409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:35.741890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:35.743629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:35.859502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:35.861079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b990591-7613-4400-8aea-430be45f40d0 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:35.880155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 729/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:37.818925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835de3-a9ce-46cf-b13a-f6464f1956b6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:37.820023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835de3-a9ce-46cf-b13a-f6464f1956b6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:37.820023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835de3-a9ce-46cf-b13a-f6464f1956b6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:37.821460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:38:37.826688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175756-f9ee-436b-9729-c8dea17f40f6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:37.827147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175756-f9ee-436b-9729-c8dea17f40f6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:37.827285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa175756-f9ee-436b-9729-c8dea17f40f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:37.893356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fa175756-f9ee-436b-9729-c8dea17f40f6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:37.894872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fa175756-f9ee-436b-9729-c8dea17f40f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:37.915217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:37.993050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:38.283046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835de3-a9ce-46cf-b13a-f6464f1956b6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:38.284610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-41835de3-a9ce-46cf-b13a-f6464f1956b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:38.386701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835de3-a9ce-46cf-b13a-f6464f1956b6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:38.452744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:38.459072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:38.459670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:38.459944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:38.460387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:38.462046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:38.580815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:38.582160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ffacb-17f1-47d6-8c7d-7341f63755fd service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:38.600192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 737/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:40.521952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4ea43-e7d8-48c7-a7d6-b86cf5de5851 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:40.523035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4ea43-e7d8-48c7-a7d6-b86cf5de5851 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:40.523035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4ea43-e7d8-48c7-a7d6-b86cf5de5851 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:40.524399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:38:40.529764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd160ae7-d6eb-48d5-8f9a-4dfc4f7c466c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:40.530219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd160ae7-d6eb-48d5-8f9a-4dfc4f7c466c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:40.530331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd160ae7-d6eb-48d5-8f9a-4dfc4f7c466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:40.599360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cd160ae7-d6eb-48d5-8f9a-4dfc4f7c466c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:38:40.600550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cd160ae7-d6eb-48d5-8f9a-4dfc4f7c466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:40.653881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:40.705660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:38:40.992406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4ea43-e7d8-48c7-a7d6-b86cf5de5851 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:38:40.994013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9bb4ea43-e7d8-48c7-a7d6-b86cf5de5851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:41.099090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4ea43-e7d8-48c7-a7d6-b86cf5de5851 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:38:41.149242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:41.175694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:38:41.176228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:38:41.176542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:38:41.176812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:38:41.178409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:38:41.288067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:38:41.289712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe706c7-ae38-49b4-86a3-6b4f377673a5 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:38:41.307639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 732/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:38:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:38:50.703646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=22 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:38:51.305336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=66 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:01.130190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be81f-fd06-4c15-8a80-26191a010356 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:01.130775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be81f-fd06-4c15-8a80-26191a010356 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:01.130877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0be81f-fd06-4c15-8a80-26191a010356 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:01.132287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 739/1471] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:01.137709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03b399-116e-4eab-a709-6bf61b550e39 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:01.138137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03b399-116e-4eab-a709-6bf61b550e39 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:01.138241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03b399-116e-4eab-a709-6bf61b550e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:01.206381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1b03b399-116e-4eab-a709-6bf61b550e39 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:01.207499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1b03b399-116e-4eab-a709-6bf61b550e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:01.302924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 733/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:01.604367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb01f1-c30b-4b08-8cfe-54b01b1e92a4 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:01.604662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb01f1-c30b-4b08-8cfe-54b01b1e92a4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:01.605487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb01f1-c30b-4b08-8cfe-54b01b1e92a4 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:01.606874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 740/1473] 10.222.0.17 () {54 vars in 967 bytes} [Wed Aug 24 16:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:01.611770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05073db7-c543-41d0-94d9-a5fd2a54c748 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:01.612054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05073db7-c543-41d0-94d9-a5fd2a54c748 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:01.612329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05073db7-c543-41d0-94d9-a5fd2a54c748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:01.687717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-05073db7-c543-41d0-94d9-a5fd2a54c748 None None] MFA Rules not processed for user `43cdc8de558d4087bed8a3bb48cf4bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:01.688936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05073db7-c543-41d0-94d9-a5fd2a54c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:01.710794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:01.767495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 734/1474] 10.222.0.17 () {58 vars in 1064 bytes} [Wed Aug 24 16:39:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:01.776285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb01f1-c30b-4b08-8cfe-54b01b1e92a4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:01.778090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-58eb01f1-c30b-4b08-8cfe-54b01b1e92a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:01.887344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb01f1-c30b-4b08-8cfe-54b01b1e92a4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:01.961582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:01.962187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:01.962468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:01.962742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:01.964332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:02.017765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:02.073536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:02.074982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd78359-cd51-4b40-89ef-d222960433b5 service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:02.097646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 741/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:04.114062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9fa67-b5e8-4550-8d16-ef8959678c13 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:04.115252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9fa67-b5e8-4550-8d16-ef8959678c13 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:04.115429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9fa67-b5e8-4550-8d16-ef8959678c13 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:04.116839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 735/1476] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:04.122306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ad3d7-0012-4098-aa36-5850d0105675 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:04.122746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ad3d7-0012-4098-aa36-5850d0105675 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:04.122979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ad3d7-0012-4098-aa36-5850d0105675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:04.189988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-404ad3d7-0012-4098-aa36-5850d0105675 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:04.191124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-404ad3d7-0012-4098-aa36-5850d0105675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:04.289178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 742/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:04.571870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9fa67-b5e8-4550-8d16-ef8959678c13 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:04.573470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-09b9fa67-b5e8-4550-8d16-ef8959678c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:04.699934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9fa67-b5e8-4550-8d16-ef8959678c13 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:04.750552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:04.800172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:04.801263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:04.801728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:04.802220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:04.803081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:04.804927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:39:04.804927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:39:04.804927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:39:04.806074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac00bb-60cd-45f2-ac72-cc49dac9690b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:04.826191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 736/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Aug 24 16:39:04 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:39:06.825385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13050072-a1c3-4b9f-87da-ba5dc553d788 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:06.825898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13050072-a1c3-4b9f-87da-ba5dc553d788 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:06.825898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13050072-a1c3-4b9f-87da-ba5dc553d788 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:06.827190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 743/1479] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:06.832300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf475-bed4-49f8-9329-400111a66d52 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:06.832757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf475-bed4-49f8-9329-400111a66d52 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:06.832920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abf475-bed4-49f8-9329-400111a66d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:06.900629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-05abf475-bed4-49f8-9329-400111a66d52 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:06.901307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05abf475-bed4-49f8-9329-400111a66d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:06.999210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 737/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:07.300576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13050072-a1c3-4b9f-87da-ba5dc553d788 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:07.302696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-13050072-a1c3-4b9f-87da-ba5dc553d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:07.312311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:07.447363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13050072-a1c3-4b9f-87da-ba5dc553d788 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:07.527605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:07.561452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:07.562059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:07.562335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:07.562602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:07.564198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:07.653624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:07.655212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893ffbe-7716-406c-8f29-dcacf6ba8e8d service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:07.674014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 744/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:09.835656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90176-9690-410a-8831-cb9faf95dbdf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:09.836276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90176-9690-410a-8831-cb9faf95dbdf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:09.836276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d90176-9690-410a-8831-cb9faf95dbdf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:09.837561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 738/1482] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:09.842882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee19b-fa8c-4c37-9358-59a83cdb89d7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:09.843289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee19b-fa8c-4c37-9358-59a83cdb89d7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:09.843431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee19b-fa8c-4c37-9358-59a83cdb89d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:09.913446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0baee19b-fa8c-4c37-9358-59a83cdb89d7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:09.914662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0baee19b-fa8c-4c37-9358-59a83cdb89d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:09.961628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:10.011652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 745/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:10.301889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90176-9690-410a-8831-cb9faf95dbdf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:10.303426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-55d90176-9690-410a-8831-cb9faf95dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:10.327385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:10.409287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d90176-9690-410a-8831-cb9faf95dbdf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:10.483802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:10.484432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:10.484708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:10.484972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:10.486559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:10.682123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:10.717205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:10.718649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a927dc-1c07-4b29-aa44-18ac8026f41c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:10.737278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 739/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:12.721509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0677be-b38d-4edc-b2eb-8e77e157f2fb None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.722109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0677be-b38d-4edc-b2eb-8e77e157f2fb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.722109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0677be-b38d-4edc-b2eb-8e77e157f2fb None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.724008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 746/1485] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:12.729915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ab3d-8988-4269-b796-4254e3278ead None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:12.730369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ab3d-8988-4269-b796-4254e3278ead None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:12.730472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5ab3d-8988-4269-b796-4254e3278ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:12.797943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-96c5ab3d-8988-4269-b796-4254e3278ead None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:12.799138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-96c5ab3d-8988-4269-b796-4254e3278ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:12.899168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 740/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:13.197970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0677be-b38d-4edc-b2eb-8e77e157f2fb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:13.199512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9b0677be-b38d-4edc-b2eb-8e77e157f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:13.305637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0677be-b38d-4edc-b2eb-8e77e157f2fb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:13.377829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:13.379758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:13.380387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:13.380670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:13.380940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:13.382544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:13.497774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:13.499387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e01a5-e424-4560-843b-0fe43e8abeb5 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:13.528318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 747/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:15.613379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546389a-699d-4ba1-b714-4b256ed5184b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.614495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546389a-699d-4ba1-b714-4b256ed5184b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.614495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546389a-699d-4ba1-b714-4b256ed5184b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.615886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 741/1488] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:15.621136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5c482-5110-4c08-93f2-e7cd1379b38b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:15.621541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5c482-5110-4c08-93f2-e7cd1379b38b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:15.621691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5c482-5110-4c08-93f2-e7cd1379b38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:15.695112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b2b5c482-5110-4c08-93f2-e7cd1379b38b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:15.696326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b2b5c482-5110-4c08-93f2-e7cd1379b38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:15.748854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:15.794172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 748/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:16.084083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546389a-699d-4ba1-b714-4b256ed5184b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:16.085649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0546389a-699d-4ba1-b714-4b256ed5184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:16.121188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:16.192288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546389a-699d-4ba1-b714-4b256ed5184b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:16.266374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:16.266971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:16.267244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:16.267766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:16.269377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:16.349682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:16.382166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:16.383609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366ddab-6217-4021-bcca-776e0a123a0a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:16.403949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 742/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:18.721702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aa280-e0fa-451a-a1f1-0c15e78399ea None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:18.722240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aa280-e0fa-451a-a1f1-0c15e78399ea None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:18.722240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aa280-e0fa-451a-a1f1-0c15e78399ea None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:18.723839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 749/1491] 10.222.0.17 () {54 vars in 927 bytes} [Wed Aug 24 16:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:39:18.729229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb710-9140-4f08-b862-3c8dffcd0e4c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:18.729668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb710-9140-4f08-b862-3c8dffcd0e4c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:18.729767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb710-9140-4f08-b862-3c8dffcd0e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:18.795032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1f7bb710-9140-4f08-b862-3c8dffcd0e4c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:39:18.796216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1f7bb710-9140-4f08-b862-3c8dffcd0e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:18.891245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 743/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Aug 24 16:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:39:19.180381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aa280-e0fa-451a-a1f1-0c15e78399ea None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:39:19.181914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7f5aa280-e0fa-451a-a1f1-0c15e78399ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:19.284536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aa280-e0fa-451a-a1f1-0c15e78399ea None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:39:19.359748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:39:19.360361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:39:19.360657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:39:19.360923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:39:19.362513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:39:19.372230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:19.471287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:39:19.472720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31332e-8967-4d3a-a69d-ebfde69d41a3 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:39:19.491231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 750/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:39:28.889438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=82 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:39:29.488304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=50 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:39.569115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.569712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.569712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.646125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:39.647335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:39.743399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 744/1494] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:39.752196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:39.752704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:39.752811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:39.828552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:39.829680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:39.928234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 751/1495] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:40:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:39.933107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:39.934348 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:39.946462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88888cb-425e-4c52-a7aa-7301b1396785 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.009687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:40.042625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.043333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.043496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.043763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.044434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:40:40.044737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.045837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:40.045837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:40:40.045837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:40:40.047500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.059096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 745/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:40:40.068089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.079401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca7757-68da-418f-8076-51fd2defe6c0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.141557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:40.175631 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.176231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.176535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.176802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.177451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.179144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:40.179144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:40:40.179144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:40:40.180379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.307512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 752/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:40:40 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:40.312168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.313906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.347019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14fe06-911a-43bb-a322-7ebe7584a187 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.447610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.448205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.448508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.448771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.449391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.450404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:40.450404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:40:40.450404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:40:40.451627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.574326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 746/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:40:40 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:40:40.578569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.580083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.642670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e06ea-eba9-4af7-b9c3-352387dbae57 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.725261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:40.736232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.736857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.737132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.737398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.738030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.739054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:40.739054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:40:40.739054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:40:40.739936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.752637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 753/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:40.756886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.758404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.770022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090c5c-d54c-4ac5-91da-35cebb7a6c74 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:40.794399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:40.864516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:40.865120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:40.865392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:40.865654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:40.866289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:40.867309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:40.867309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:40:40.867309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:40:40.868186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e97424-19e8-476a-888f-90326e2a88f5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:40.880590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 747/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:40:40.884918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:40.886549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:40.920616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd9134-480a-4b1b-95fe-0b3db6f4a7df admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.013923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.014575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.015132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.015948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] PATH_INFO: `/v3/projects/6318030e995048419f4420fe05a164f7/users/340c582bf39542dfb0c5284ae2bb445b/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.038078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6318030e995048419f4420fe05a164f7, user_id=340c582bf39542dfb0c5284ae2bb445b, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.039104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:41.039104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:40:41.039104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:40:41.041099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a53f0-47e9-49e7-9648-0998519f4df3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.121692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 754/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:40:40 2022] PUT /identity/v3/projects/6318030e995048419f4420fe05a164f7/users/340c582bf39542dfb0c5284ae2bb445b/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:41.134081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203dfa3-5755-46e5-8a56-07a0fcf4ecd5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.134365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203dfa3-5755-46e5-8a56-07a0fcf4ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.134641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203dfa3-5755-46e5-8a56-07a0fcf4ecd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.191909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c203dfa3-5755-46e5-8a56-07a0fcf4ecd5 None None] MFA Rules not processed for user `340c582bf39542dfb0c5284ae2bb445b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:41.193076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c203dfa3-5755-46e5-8a56-07a0fcf4ecd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.271121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 748/1502] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:41.284894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc97ae-9676-4766-a127-54196efa88ec None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.285177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc97ae-9676-4766-a127-54196efa88ec None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.285478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc97ae-9676-4766-a127-54196efa88ec None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.286807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 755/1503] 10.222.0.17 () {54 vars in 964 bytes} [Wed Aug 24 16:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:41.291580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a76f0-f16e-4873-b047-0ee88411e683 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.291875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a76f0-f16e-4873-b047-0ee88411e683 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.292151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a76f0-f16e-4873-b047-0ee88411e683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.323771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:41.369390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-478a76f0-f16e-4873-b047-0ee88411e683 None None] MFA Rules not processed for user `ef06aba40382479289b1e9e19395a2be`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:41.370605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-478a76f0-f16e-4873-b047-0ee88411e683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.450628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 749/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Aug 24 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:41.458843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc97ae-9676-4766-a127-54196efa88ec None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.460413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-44cc97ae-9676-4766-a127-54196efa88ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.511137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:41.566570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc97ae-9676-4766-a127-54196efa88ec None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:41.641428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:41.642019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.642292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.642556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.644124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.748230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:41.750214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:41.753744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:41.769443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 756/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:41.814491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaefade-33e6-4420-8443-1846b15b82bf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:41.814931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaefade-33e6-4420-8443-1846b15b82bf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:41.815041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaefade-33e6-4420-8443-1846b15b82bf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:41.816258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 750/1506] 10.222.0.17 () {52 vars in 830 bytes} [Wed Aug 24 16:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:40:41.819855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:41.821095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:41.922779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebfa51-9bef-49ff-a93b-796e2fa31e57 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.016837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.017382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.017655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.017920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] PATH_INFO: `/v3/users/340c582bf39542dfb0c5284ae2bb445b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.029949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=340c582bf39542dfb0c5284ae2bb445b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.031093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:42.031093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:40:42.031093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:40:42.032305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.041795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:42.193539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340c582bf39542dfb0c5284ae2bb445b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.194107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340c582bf39542dfb0c5284ae2bb445b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.208596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340c582bf39542dfb0c5284ae2bb445b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.263614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340c582bf39542dfb0c5284ae2bb445b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.264056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340c582bf39542dfb0c5284ae2bb445b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:42.292759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 757/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:40:41 2022] DELETE /identity/v3/users/340c582bf39542dfb0c5284ae2bb445b => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:42.297703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.297983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.298252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.354056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:40:42.355368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.431330 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:42.443433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 751/1508] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:40:42.453252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:42.454497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.558131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14271e05-e67b-4efd-ba6e-356027522e0e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:42.612723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:42.631125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:42.631731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:42.632009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:42.632280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:42.633891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:42.745211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:42.746860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00fddf-34a1-40e4-87fa-3e5f1f08bba1 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:42.767952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 758/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:40:43.248801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:40:43.251282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:40:43.261923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d656af-4728-4146-b5bd-1cad9f402785 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:40:43.358898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:40:43.359532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:40:43.359802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:40:43.360063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] PATH_INFO: `/v3/projects/6318030e995048419f4420fe05a164f7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:40:43.369762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6318030e995048419f4420fe05a164f7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:40:43.370808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:40:43.370808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:40:43.370808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:40:43.372003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:40:43.480983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6318030e995048419f4420fe05a164f7', 'request_id': 'req-965c9f5f-ad26-4c3f-bf89-763dc5661902'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.543681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6318030e995048419f4420fe05a164f7', 'request_id': 'req-965c9f5f-ad26-4c3f-bf89-763dc5661902'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.558037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6318030e995048419f4420fe05a164f7', 'request_id': 'req-965c9f5f-ad26-4c3f-bf89-763dc5661902'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.581318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] The token cache is being invalidate because project 6318030e995048419f4420fe05a164f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:40:43.581601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.582024 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-965c9f5f-ad26-4c3f-bf89-763dc5661902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:40:43.583460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 752/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:40:43 2022] DELETE /identity/v3/projects/6318030e995048419f4420fe05a164f7 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:40:52.764977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=62 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:40:53.579149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:14.418338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506cb1-11df-446c-902f-0728692b5db1 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.418975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506cb1-11df-446c-902f-0728692b5db1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.418975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a506cb1-11df-446c-902f-0728692b5db1 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.420214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:14.425095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda47d6d-e615-4d6b-bb22-e8c4ce4c9013 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.425515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda47d6d-e615-4d6b-bb22-e8c4ce4c9013 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.425668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda47d6d-e615-4d6b-bb22-e8c4ce4c9013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.508782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-eda47d6d-e615-4d6b-bb22-e8c4ce4c9013 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:14.510001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eda47d6d-e615-4d6b-bb22-e8c4ce4c9013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:14.612744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:14.768044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a506cb1-11df-446c-902f-0728692b5db1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:14.770198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3a506cb1-11df-446c-902f-0728692b5db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:14.892117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a506cb1-11df-446c-902f-0728692b5db1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:14.899962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9021-768c-4354-970f-e8604be80a64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:14.900630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9021-768c-4354-970f-e8604be80a64 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:14.901057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9021-768c-4354-970f-e8604be80a64 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:14.901329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9021-768c-4354-970f-e8604be80a64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:14.910333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c9021-768c-4354-970f-e8604be80a64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:14.911940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c9021-768c-4354-970f-e8604be80a64 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:14.921361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 760/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:15.769636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc022-3a56-48f9-86e5-789fa27efcee None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:15.770214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc022-3a56-48f9-86e5-789fa27efcee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:15.770214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc022-3a56-48f9-86e5-789fa27efcee None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:15.771500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:15.776199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19d0d8-2263-4a94-83ca-8d60a9f26c4f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:15.776652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19d0d8-2263-4a94-83ca-8d60a9f26c4f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:15.776784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19d0d8-2263-4a94-83ca-8d60a9f26c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:15.795598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3d19d0d8-2263-4a94-83ca-8d60a9f26c4f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:15.797314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3d19d0d8-2263-4a94-83ca-8d60a9f26c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:15.805255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:15.957479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc022-3a56-48f9-86e5-789fa27efcee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:15.972027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc022-3a56-48f9-86e5-789fa27efcee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:15.977411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2c2b-8bd3-40b7-8518-02de3d478544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:15.978011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2c2b-8bd3-40b7-8518-02de3d478544 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:15.978280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2c2b-8bd3-40b7-8518-02de3d478544 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:15.978543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2c2b-8bd3-40b7-8518-02de3d478544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:15.987178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2c2b-8bd3-40b7-8518-02de3d478544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:15.988817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2c2b-8bd3-40b7-8518-02de3d478544 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:15.998054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 755/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:16.483634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9a95-eb83-437b-8a1d-cc800d699406 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:16.483847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9a95-eb83-437b-8a1d-cc800d699406 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:16.484657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d9a95-eb83-437b-8a1d-cc800d699406 None None] PATH_INFO: `/v3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:16.486141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 762/1517] 10.222.0.40 () {54 vars in 925 bytes} [Wed Aug 24 16:42:16 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:42:16.492320 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a357468-9c58-4d21-9c86-e82941161d92 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:16.492630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a357468-9c58-4d21-9c86-e82941161d92 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:16.492904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a357468-9c58-4d21-9c86-e82941161d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:16.552555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1a357468-9c58-4d21-9c86-e82941161d92 None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:16.553775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1a357468-9c58-4d21-9c86-e82941161d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:16.570930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:16.628940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 756/1518] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 24 16:42:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:42:20.542526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bd47f-115e-4c51-bc84-315c1cf558e2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:20.543050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bd47f-115e-4c51-bc84-315c1cf558e2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:20.543050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bd47f-115e-4c51-bc84-315c1cf558e2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:20.544445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 763/1519] 10.222.0.17 () {54 vars in 961 bytes} [Wed Aug 24 16:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:20.550885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b16db-01ce-4eaa-a702-0d4c92a120ba None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:20.551354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b16db-01ce-4eaa-a702-0d4c92a120ba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:20.551498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b16db-01ce-4eaa-a702-0d4c92a120ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:20.572419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d49b16db-01ce-4eaa-a702-0d4c92a120ba None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:20.573619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d49b16db-01ce-4eaa-a702-0d4c92a120ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:20.583027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 757/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Aug 24 16:42:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:20.590606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bd47f-115e-4c51-bc84-315c1cf558e2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:20.604951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bd47f-115e-4c51-bc84-315c1cf558e2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:20.609984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72416f31-df36-4541-8466-1cdbabb82fce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:20.610576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72416f31-df36-4541-8466-1cdbabb82fce service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:20.610854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72416f31-df36-4541-8466-1cdbabb82fce service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:20.611122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72416f31-df36-4541-8466-1cdbabb82fce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:20.619279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72416f31-df36-4541-8466-1cdbabb82fce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:20.621018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72416f31-df36-4541-8466-1cdbabb82fce service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:20.631241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 764/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:27.253623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98ff2-e634-4bdf-9f9d-5cb2aee077a9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:27.254197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98ff2-e634-4bdf-9f9d-5cb2aee077a9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:27.254197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98ff2-e634-4bdf-9f9d-5cb2aee077a9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:27.255498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:27.260243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe577d06-6107-408a-a507-2d609907c951 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:27.260678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe577d06-6107-408a-a507-2d609907c951 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:27.260835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe577d06-6107-408a-a507-2d609907c951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:27.280220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fe577d06-6107-408a-a507-2d609907c951 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:27.281446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fe577d06-6107-408a-a507-2d609907c951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:27.289843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:27.443040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98ff2-e634-4bdf-9f9d-5cb2aee077a9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:27.458329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98ff2-e634-4bdf-9f9d-5cb2aee077a9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:27.463599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1328cd3-1819-465b-bfa2-2dd691219e11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:27.464192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328cd3-1819-465b-bfa2-2dd691219e11 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:27.464504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328cd3-1819-465b-bfa2-2dd691219e11 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:27.464772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1328cd3-1819-465b-bfa2-2dd691219e11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:27.473683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1328cd3-1819-465b-bfa2-2dd691219e11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:27.475272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1328cd3-1819-465b-bfa2-2dd691219e11 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:27.484679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 759/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:28.308716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572940-d40c-44fd-8ae2-e039c05c18dd None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:28.309277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572940-d40c-44fd-8ae2-e039c05c18dd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:28.309277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572940-d40c-44fd-8ae2-e039c05c18dd None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:28.310620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 16:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:28.316179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa909c6-ee7c-4677-953e-0f7df2c7bc18 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:28.316611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa909c6-ee7c-4677-953e-0f7df2c7bc18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:28.316775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa909c6-ee7c-4677-953e-0f7df2c7bc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:28.336538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3aa909c6-ee7c-4677-953e-0f7df2c7bc18 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:28.337720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3aa909c6-ee7c-4677-953e-0f7df2c7bc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:28.345472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 16:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:28.505823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73572940-d40c-44fd-8ae2-e039c05c18dd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:28.521034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73572940-d40c-44fd-8ae2-e039c05c18dd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:28.526462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f0b58-439a-4214-8075-dae7fe4de929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:28.527080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f0b58-439a-4214-8075-dae7fe4de929 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:28.527356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f0b58-439a-4214-8075-dae7fe4de929 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:28.527624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f0b58-439a-4214-8075-dae7fe4de929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:28.535885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f0b58-439a-4214-8075-dae7fe4de929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:28.537494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f0b58-439a-4214-8075-dae7fe4de929 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:28.546676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 767/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:32.328879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c011-5ddf-4104-8d20-ab1d652e4e26 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:32.328879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c011-5ddf-4104-8d20-ab1d652e4e26 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:32.329938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141c011-5ddf-4104-8d20-ab1d652e4e26 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:32.330949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:32.336270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1efa6-3194-469d-bea7-3dfde1616956 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:32.336792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1efa6-3194-469d-bea7-3dfde1616956 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:32.336945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1efa6-3194-469d-bea7-3dfde1616956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:32.357782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-05d1efa6-3194-469d-bea7-3dfde1616956 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:32.358990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-05d1efa6-3194-469d-bea7-3dfde1616956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:32.367176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:32.661551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141c011-5ddf-4104-8d20-ab1d652e4e26 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:32.663231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7141c011-5ddf-4104-8d20-ab1d652e4e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:32.744860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141c011-5ddf-4104-8d20-ab1d652e4e26 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:32.748509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041edcb0-5163-45c6-aa21-d76cde1fc19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:32.749107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041edcb0-5163-45c6-aa21-d76cde1fc19a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:32.749382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041edcb0-5163-45c6-aa21-d76cde1fc19a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:32.749651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041edcb0-5163-45c6-aa21-d76cde1fc19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:32.759314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041edcb0-5163-45c6-aa21-d76cde1fc19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:32.760793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041edcb0-5163-45c6-aa21-d76cde1fc19a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:32.770455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 762/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:34.719583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239145c-1144-4a5b-b775-ec9cf3beec41 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:34.720692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239145c-1144-4a5b-b775-ec9cf3beec41 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:34.720786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239145c-1144-4a5b-b775-ec9cf3beec41 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:34.722142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:34.727423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f85fc-a84c-4cc5-909c-fcb9f2f73ef0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:34.727822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f85fc-a84c-4cc5-909c-fcb9f2f73ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:34.727965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f85fc-a84c-4cc5-909c-fcb9f2f73ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:34.747820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-149f85fc-a84c-4cc5-909c-fcb9f2f73ef0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:34.749024 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-149f85fc-a84c-4cc5-909c-fcb9f2f73ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:34.756974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:35.044272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239145c-1144-4a5b-b775-ec9cf3beec41 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:35.055115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239145c-1144-4a5b-b775-ec9cf3beec41 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:35.058526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12c092-ab40-4d47-a19e-4051d0b8714e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:35.059154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12c092-ab40-4d47-a19e-4051d0b8714e service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:35.059431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12c092-ab40-4d47-a19e-4051d0b8714e service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:35.059701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12c092-ab40-4d47-a19e-4051d0b8714e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:35.068704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12c092-ab40-4d47-a19e-4051d0b8714e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:35.070123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12c092-ab40-4d47-a19e-4051d0b8714e service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:35.080771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 770/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:38.611173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97db61-4d96-485f-881e-5845b0b94f94 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:38.611743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97db61-4d96-485f-881e-5845b0b94f94 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:38.611743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97db61-4d96-485f-881e-5845b0b94f94 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:38.613095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:38.618295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80209a72-056d-451c-8531-c0e57f1ed78e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:38.618723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80209a72-056d-451c-8531-c0e57f1ed78e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:38.618834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80209a72-056d-451c-8531-c0e57f1ed78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:38.638714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-80209a72-056d-451c-8531-c0e57f1ed78e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:38.639904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-80209a72-056d-451c-8531-c0e57f1ed78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:38.648318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:39.192530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97db61-4d96-485f-881e-5845b0b94f94 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:39.194740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2d97db61-4d96-485f-881e-5845b0b94f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:39.267315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:39.280441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97db61-4d96-485f-881e-5845b0b94f94 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:39.284714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c7d22-49cb-4311-9945-bf027d0f7841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:39.285322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c7d22-49cb-4311-9945-bf027d0f7841 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:39.285601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c7d22-49cb-4311-9945-bf027d0f7841 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:39.285872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c7d22-49cb-4311-9945-bf027d0f7841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:39.294837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c7d22-49cb-4311-9945-bf027d0f7841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:39.296304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c7d22-49cb-4311-9945-bf027d0f7841 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:39.306679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 765/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:41.229142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347105-33dc-4417-99c7-71cddb308326 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:41.229971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347105-33dc-4417-99c7-71cddb308326 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:41.229971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347105-33dc-4417-99c7-71cddb308326 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:41.231204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 16:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:42:41.237750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25cfa-b49e-489d-8a55-3bd3b9fba820 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:41.238234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25cfa-b49e-489d-8a55-3bd3b9fba820 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:41.238363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25cfa-b49e-489d-8a55-3bd3b9fba820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:41.258490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-65d25cfa-b49e-489d-8a55-3bd3b9fba820 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:42:41.259690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-65d25cfa-b49e-489d-8a55-3bd3b9fba820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:41.267942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 16:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:42:41.832873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347105-33dc-4417-99c7-71cddb308326 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:42:41.834566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-18347105-33dc-4417-99c7-71cddb308326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:42:41.848203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347105-33dc-4417-99c7-71cddb308326 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:42:41.851792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82686d-ba4c-4036-a599-dce073e353cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:42:41.852413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82686d-ba4c-4036-a599-dce073e353cb service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:42:41.852710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82686d-ba4c-4036-a599-dce073e353cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:42:41.852961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82686d-ba4c-4036-a599-dce073e353cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:42:41.862075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82686d-ba4c-4036-a599-dce073e353cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:42:41.863506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82686d-ba4c-4036-a599-dce073e353cb service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:42:41.872862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 773/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:42:51.253732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=11 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:42:51.868674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:05.751924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:05.753103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:05.753616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:05.781979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:05.783592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:05.796459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 767/1540] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:05.805764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:05.806316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:05.806316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:05.843196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:05.843196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:05.858013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 774/1541] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:05.862622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:05.877891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2510-866c-4db8-9fde-673757494e2b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:05.905088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:05.906018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:05.906518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:05.907006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:05.907964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:05.908505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:05.910497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e6f00-2504-424a-ad16-145ad83dd756 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:05.928995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 768/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:05.938500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:05.955300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7c24-19c7-4755-b0bf-341cca4ce8eb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:05.957135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f1a0e-925c-4942-bb9a-c74af85be9b3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:05.961754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f1a0e-925c-4942-bb9a-c74af85be9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:05.961754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f1a0e-925c-4942-bb9a-c74af85be9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:05.969223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc4ac9-c6a7-4187-a051-7f5270aa1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:05.969862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4ac9-c6a7-4187-a051-7f5270aa1d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:05.970140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4ac9-c6a7-4187-a051-7f5270aa1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:05.970412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc4ac9-c6a7-4187-a051-7f5270aa1d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:05.971309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc4ac9-c6a7-4187-a051-7f5270aa1d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:05.974236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc4ac9-c6a7-4187-a051-7f5270aa1d1f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:05.993561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9b4f1a0e-925c-4942-bb9a-c74af85be9b3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:05.995173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9b4f1a0e-925c-4942-bb9a-c74af85be9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.010447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 769/1543] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:06.014271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef144912-f475-4315-8ca9-09dc92155785 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.014824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef144912-f475-4315-8ca9-09dc92155785 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.015325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef144912-f475-4315-8ca9-09dc92155785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.054279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ef144912-f475-4315-8ca9-09dc92155785 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.056750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ef144912-f475-4315-8ca9-09dc92155785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.071170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 770/1544] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:06.072810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28f6ed-fd9e-48b3-8c14-87c61dcb295e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.073290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28f6ed-fd9e-48b3-8c14-87c61dcb295e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.074535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28f6ed-fd9e-48b3-8c14-87c61dcb295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.107867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 775/1545] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:05 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:06.109184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082861d5-9935-473b-b20b-f667abd6b1d6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.109468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082861d5-9935-473b-b20b-f667abd6b1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.109744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082861d5-9935-473b-b20b-f667abd6b1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.110797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dd28f6ed-fd9e-48b3-8c14-87c61dcb295e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.114345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dd28f6ed-fd9e-48b3-8c14-87c61dcb295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.144303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-082861d5-9935-473b-b20b-f667abd6b1d6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.145600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-082861d5-9935-473b-b20b-f667abd6b1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.182332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 771/1546] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.187200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.187200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.187200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.203851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.205084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.209956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 776/1547] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.211137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dfdc5-954c-44c6-962a-a2137411cdf1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.211424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dfdc5-954c-44c6-962a-a2137411cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.211697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dfdc5-954c-44c6-962a-a2137411cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.216837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 772/1548] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.216837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.226209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b04214-1011-408c-ae74-a2e771db4269 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.232133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.232767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.233048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.233317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.233941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.234383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-584dfdc5-954c-44c6-962a-a2137411cdf1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.235592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-584dfdc5-954c-44c6-962a-a2137411cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.235874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.243437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 777/1549] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.244620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00f60e-5d53-4744-9e0a-3b2d9f6d6635 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.245227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00f60e-5d53-4744-9e0a-3b2d9f6d6635 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.245227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00f60e-5d53-4744-9e0a-3b2d9f6d6635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.264859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6f00f60e-5d53-4744-9e0a-3b2d9f6d6635 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.266053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6f00f60e-5d53-4744-9e0a-3b2d9f6d6635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.274018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 778/1550] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.275181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea2b6e-2bbf-460f-bffa-72970bdd9460 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.275461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea2b6e-2bbf-460f-bffa-72970bdd9460 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.275731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea2b6e-2bbf-460f-bffa-72970bdd9460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.295338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-84ea2b6e-2bbf-460f-bffa-72970bdd9460 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.296582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-84ea2b6e-2bbf-460f-bffa-72970bdd9460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.304554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 779/1551] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.305695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.305973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.306241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.316213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 773/1552] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:06.317080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.327524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.328753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.329579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bcdd-353b-40b6-8c74-b72bdabf5ae7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.334596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.335198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.335474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.335745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.336434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.336738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.336978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 780/1553] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.337861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.338426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b09905-ecd2-4d0b-8ced-c9559bee6362 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.347411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66b474-9673-491c-bfb6-157ebef3d08f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.352532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.353190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.353715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.353881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.354502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.354803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.356481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:06.356481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:44:06.356481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:44:06.356934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 774/1554] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.357437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db3933-57a2-4f5d-a08d-ab67cd5a8c04 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.358176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069178a-e729-4165-93f0-78a30e6c1221 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.358457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069178a-e729-4165-93f0-78a30e6c1221 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.358729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069178a-e729-4165-93f0-78a30e6c1221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.384524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 781/1555] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.384769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4069178a-e729-4165-93f0-78a30e6c1221 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.384885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4c0ac-642b-48fc-a95e-502c169ad2a0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.384885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4c0ac-642b-48fc-a95e-502c169ad2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.384885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4c0ac-642b-48fc-a95e-502c169ad2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.385095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4069178a-e729-4165-93f0-78a30e6c1221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.388399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 775/1556] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.389630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.389898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.390164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.403120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-86c4c0ac-642b-48fc-a95e-502c169ad2a0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.404472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-86c4c0ac-642b-48fc-a95e-502c169ad2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.409721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.410891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.412787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 782/1557] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.416820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984b02-389b-4905-b92e-9c0365928832 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.416820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984b02-389b-4905-b92e-9c0365928832 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.416820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f984b02-389b-4905-b92e-9c0365928832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.422191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 776/1558] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.422993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.432288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc454a68-4ee9-4ae3-bb47-d385f9be3b7b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.437945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.438439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2f984b02-389b-4905-b92e-9c0365928832 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.438572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.438828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.439117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.439718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2f984b02-389b-4905-b92e-9c0365928832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.439828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.441371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.448054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 783/1559] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.453019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.453019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.453019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.453278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 777/1560] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:06.455791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.467003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6fe02-3120-47b7-9526-f2838d1a65b8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.470313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:06.471450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.471593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:06.472036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.472308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.472605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.473214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.474233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:06.474233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:44:06.474233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:44:06.475405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.480874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 784/1561] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:06.482255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.493097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56287420-681d-4376-8345-0e887f7d49c4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.498957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.502674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.502674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.502674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.502674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.502674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.591150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 778/1562] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:06.591973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.618018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 785/1563] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/projects => generated 361 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:06.618018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.642049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:06.658722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e51f2-105b-4081-9955-6bb2f8c190ff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.665267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:06.683275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c21cff-10ed-4a54-a42c-79b2d86f5d58 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.687949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.688578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.688861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.689131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.689778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.691356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.693003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.711813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 786/1564] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.711813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.718166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37af4b-e625-43f5-a6a5-6ac2f9799d85 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.721786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.722385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.722665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.722937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.723530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.723655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.723888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.724170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.724479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.724751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.725426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.725524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.725728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.727374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.738231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 787/1565] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.739026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.739583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 779/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.740383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.755881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc72-a1c2-4b6b-8409-bcce626b80e5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.756097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3069c91-f239-4769-999a-06501afe6502 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.760979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.761118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.761642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.761797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.761921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.762082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.762181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.762269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.762814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.762932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.763110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.764524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.764740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.774870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 780/1567] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:06.775652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.784115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 788/1568] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.784115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.785953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17445ff-79e7-4c48-8e32-8a0183c02aab admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.789327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505a512-e604-4c18-8945-24c0a2d527ef admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.790566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.791157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.791431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.791701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.792353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:06.793268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.794905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.795035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.795503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.795802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.796073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.796725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.800002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:06.800002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:44:06.800002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:44:06.800002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.806296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 781/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:06.806296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.823056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7270eb-6a03-4e15-ad24-1737a29f406d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.828101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.828726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.829004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.829274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.829896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.831814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.921187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 789/1570] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:06.921999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.931601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20098f-8222-4c2c-9afd-f52ac335d9a4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.936244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.936863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.937185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.937458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.938067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.940010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:06.950469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 782/1571] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/users => generated 281 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:06.951289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:06.961811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4311c4-736f-4cd2-b1b3-432d1b8e9f84 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:06.968112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:06.972161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:06.972161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:06.972161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:06.972161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:06.972161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.048419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 790/1572] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.049454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.071958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 783/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:06 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.071958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.115202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4929c-c523-4cde-a7cd-ed95a9d0a7a5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.146489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78db91d-e06b-4348-8811-98f570205a45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.151053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.151740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.152076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.152889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] PATH_INFO: `/v3/projects/f9652b4a38de4d4087e33ff15f193205/users/fc7edc3d8ae84e0585965afe4486bb27/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.168234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9652b4a38de4d4087e33ff15f193205, user_id=fc7edc3d8ae84e0585965afe4486bb27, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.169349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:07.169349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:44:07.169349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:44:07.173238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.173854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.174466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.174747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.175018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.175643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.177708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.237053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 784/1574] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:07 2022] PUT /identity/v3/projects/f9652b4a38de4d4087e33ff15f193205/users/fc7edc3d8ae84e0585965afe4486bb27/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:07.238092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.246319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:07.266862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 791/1575] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.269113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.272385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:07.305566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac04382-eb3c-4934-ba15-4f29f46c06d7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.334271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7390560-8ce6-49bb-819f-55e028be55ab admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.339186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.339839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.340121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.340411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.341066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.343030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.362395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.363003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.363277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.363545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.364153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.366125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.436820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 792/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.437709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.466604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 785/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/projects => generated 393 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.468153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.499983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9275078-fba9-4331-a648-39a6fd3540ba admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.535067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56071e6a-72cf-41b1-904f-1ea8e8915e58 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.539835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.540552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.540854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.541121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.541792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.543455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.553255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 786/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:07.554092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.560171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.560835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.561113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.561501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.562264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.563894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.566393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9938642-9870-466f-bd79-439591f56aee admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.570920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f093351-d981-4d72-bc9f-66b2e4b88fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.571499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f093351-d981-4d72-bc9f-66b2e4b88fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.571771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f093351-d981-4d72-bc9f-66b2e4b88fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.572037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f093351-d981-4d72-bc9f-66b2e4b88fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.572702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f093351-d981-4d72-bc9f-66b2e4b88fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.574521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 793/1579] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:07.575342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.577712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f093351-d981-4d72-bc9f-66b2e4b88fb8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.584678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7bc14-92dd-4443-9133-7569d0bc7698 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.589270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.590133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.590469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.590739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.591366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.593280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.687256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 787/1580] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.688826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.689165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.689617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.709898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 794/1581] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.713632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.718985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] MFA Rules not processed for user `fc7edc3d8ae84e0585965afe4486bb27`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:07.720200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:07.720392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0194aa-4275-4a71-b6e9-5a5d6e1c52cd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.724424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:07.725022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.725644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.725911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.726183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.726843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.728828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.740878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:07.784829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 788/1582] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:07.785612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.796027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453604c-ec79-43a3-9da8-0a90cbbe9eae None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.800608 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.801199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.801470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.801763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.802379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.804302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.875947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 795/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.876887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.886819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd83d1d-1d6a-404b-bb0c-794f94604f31 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.891488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.892075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.892376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.892649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.893263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.895187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:07.973284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 789/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:07.974282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:07.984547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1589fca-74df-46c8-92df-cfd2e318485a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:07.989061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:07.989649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:07.989921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:07.990450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:07.991089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:07.992732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.004911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 790/1585] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.005781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.006990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 796/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:07 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:08.007887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.016566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca42b29-bc3b-4426-a6e0-3cef833b913f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.019343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ac296-e09e-4666-a19b-b8fe0efe5dd1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.021650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.022304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.022575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.022842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.023480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.024013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33a797-05a6-4936-94a0-60c71ebb5dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.024638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33a797-05a6-4936-94a0-60c71ebb5dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.024921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33a797-05a6-4936-94a0-60c71ebb5dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.025117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.025226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33a797-05a6-4936-94a0-60c71ebb5dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.025804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33a797-05a6-4936-94a0-60c71ebb5dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.027890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33a797-05a6-4936-94a0-60c71ebb5dbf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.036804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 791/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.037757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 797/1588] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.038101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.038974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0e968-1daf-40b5-9af6-6029a176826c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.039240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0e968-1daf-40b5-9af6-6029a176826c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.039505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0e968-1daf-40b5-9af6-6029a176826c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.040741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 798/1589] 10.222.0.17 () {54 vars in 962 bytes} [Wed Aug 24 16:44:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:08.041427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0e968-1daf-40b5-9af6-6029a176826c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.051878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298093fa-d883-4555-bd43-b4fb28330c6a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.054361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0e968-1daf-40b5-9af6-6029a176826c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.056675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.057310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.057613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.057880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.058531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.059006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.059663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.060022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.060145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.060300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.060953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.062548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.070619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 792/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.072446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.075877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 799/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.075877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.084499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd47fa-39cc-4db7-86e7-8a006fcdc8d1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.085767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31e4d3-6634-410e-b440-f8edb64149c3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.089849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.090572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.090923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.091298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.091517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.091993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.092121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.092425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.092699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.093338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.093656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.094935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.104195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 800/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.105617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.107938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 793/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.108150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.115247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6599d-d68e-4334-be05-f40f070dcb42 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.120965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe16a2-7a4d-4c28-a4cc-0faa8c01a79c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.121138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.121138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.121138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.121437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] PATH_INFO: `/v3/projects/2ac68a42d7514dc8af4c0cdc1d423f8e/users/7cc997ec4a3f460197d19cedb78cfee8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.123448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea07f7d-023b-432a-ac66-7b59c04ddcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.124931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea07f7d-023b-432a-ac66-7b59c04ddcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.124931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea07f7d-023b-432a-ac66-7b59c04ddcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.124931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea07f7d-023b-432a-ac66-7b59c04ddcdb admin admin] PATH_INFO: `/v3/projects/8af535d51fd345f8a9584cc6aaa38bda/users/1d9f7755863d45bfb04a1498bd62e367/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.136399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac68a42d7514dc8af4c0cdc1d423f8e, user_id=7cc997ec4a3f460197d19cedb78cfee8, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.139201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.141005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea07f7d-023b-432a-ac66-7b59c04ddcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af535d51fd345f8a9584cc6aaa38bda, user_id=1d9f7755863d45bfb04a1498bd62e367, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.145008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea07f7d-023b-432a-ac66-7b59c04ddcdb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.206380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 801/1594] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/2ac68a42d7514dc8af4c0cdc1d423f8e/users/7cc997ec4a3f460197d19cedb78cfee8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:08.207315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.223736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 794/1595] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/8af535d51fd345f8a9584cc6aaa38bda/users/1d9f7755863d45bfb04a1498bd62e367/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:08.225010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.225328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.225631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.264287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:08.265558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:08.269862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbddc9f-a39c-4679-9b15-70b168d30961 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.322791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:08.329266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.334264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.334264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.334264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.334264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.334264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.334801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 795/1596] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Aug 24 16:44:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:08.334801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.344486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 802/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.345417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.345541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc62cf-2d08-438b-a050-eb6faa0eb196 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.350240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.350847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.351136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.351393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.352061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.353725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.356310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199499-a865-44ca-95aa-695adace2a97 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.360967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.361581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.361862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.362133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.362763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.364372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.365186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 796/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.365985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.374715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 803/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.375588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.376926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b987b22-6ff0-4fb1-a310-8f505540962d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.382407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d5536-48bd-4d6d-b6d1-d67a883aea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.383490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5536-48bd-4d6d-b6d1-d67a883aea45 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.383733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5536-48bd-4d6d-b6d1-d67a883aea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.384010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5536-48bd-4d6d-b6d1-d67a883aea45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.384693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5536-48bd-4d6d-b6d1-d67a883aea45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.386815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe464e24-c890-402e-b08e-1b5d47c6cd92 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.389514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5536-48bd-4d6d-b6d1-d67a883aea45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.392334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb79b5-ce65-442d-a52b-0aa880c093ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.392957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb79b5-ce65-442d-a52b-0aa880c093ef admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.393253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb79b5-ce65-442d-a52b-0aa880c093ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.393504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb79b5-ce65-442d-a52b-0aa880c093ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.396013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb79b5-ce65-442d-a52b-0aa880c093ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.396013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb79b5-ce65-442d-a52b-0aa880c093ef admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.401489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 797/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.403451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.403733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.404100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.411279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 804/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:08.412491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.412775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.413046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.420685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:08.432406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:08.433674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:08.437626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] MFA Rules not processed for user `7cc997ec4a3f460197d19cedb78cfee8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:08.438967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:08.441333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 805/1602] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:08.445295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.459339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18c975-832a-407f-aef0-c5cabc21ff91 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.465718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.466317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.466595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.466864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.505104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 798/1603] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:08.505898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.517322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6283e-fb6b-40c7-be4e-94903c93df56 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.521997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.522651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.522919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.523185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] PATH_INFO: `/v3/projects/6dd83e5ccf51423e876b0dbe1b661bc5/users/18df998dc1cb47d5b5a860bbc21cf705/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.525723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.528218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.537698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 806/1604] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:08.538590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.541194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd83e5ccf51423e876b0dbe1b661bc5, user_id=18df998dc1cb47d5b5a860bbc21cf705, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.544982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.548638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3856a0-0202-4401-a10e-cefa54a1f149 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.553429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.554078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.554377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.554651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] PATH_INFO: `/v3/projects/962fd0e297b24ea2a77441d8d8928d5b/users/1abbb93d22624eaa8a14f7533b485e81/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.569232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=962fd0e297b24ea2a77441d8d8928d5b, user_id=1abbb93d22624eaa8a14f7533b485e81, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.571943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.654573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 799/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/6dd83e5ccf51423e876b0dbe1b661bc5/users/18df998dc1cb47d5b5a860bbc21cf705/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:08.666285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.689001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 807/1606] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/962fd0e297b24ea2a77441d8d8928d5b/users/1abbb93d22624eaa8a14f7533b485e81/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:08.691328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.751200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f72b91-4c59-4a86-9bef-5a9871523a07 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.772211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9c6b7-7129-4779-b823-aac467914c1f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.777476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.778226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.778511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.778805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] PATH_INFO: `/v3/projects/c99d97707cc2485284d8d33522ebb918/users/7be50b041b5947d5909ed6ce516ec8bf/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.794838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99d97707cc2485284d8d33522ebb918, user_id=7be50b041b5947d5909ed6ce516ec8bf, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.797544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.798757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:08.810331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.812430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.812430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:08.812430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] PATH_INFO: `/v3/projects/d09164dc6ad44a87b7fae777d9f8435a/users/491e2b9a44bd47b190b1a0b1250bf1f3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:08.825431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09164dc6ad44a87b7fae777d9f8435a, user_id=491e2b9a44bd47b190b1a0b1250bf1f3, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:08.828110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:08.898650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 808/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/c99d97707cc2485284d8d33522ebb918/users/7be50b041b5947d5909ed6ce516ec8bf/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:08.899464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.938500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 800/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/d09164dc6ad44a87b7fae777d9f8435a/users/491e2b9a44bd47b190b1a0b1250bf1f3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:08.938500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:08.953510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:08.963938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa05cdf-5aed-431a-acba-a1fd5bb6707c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.995083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf6402-e626-4e61-b02d-68640707aeeb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:08.999058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:08.999704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:08.999981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.000244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.023722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae4185-50d6-42f2-bcc0-1342f3c375ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.024519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae4185-50d6-42f2-bcc0-1342f3c375ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.024804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae4185-50d6-42f2-bcc0-1342f3c375ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.025083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae4185-50d6-42f2-bcc0-1342f3c375ae admin admin] PATH_INFO: `/v3/projects/b71890b7a23944ee939c4bd13408063f/users/4ce12f8401f3479ba14e5e1c3133a71d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.040857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae4185-50d6-42f2-bcc0-1342f3c375ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71890b7a23944ee939c4bd13408063f, user_id=4ce12f8401f3479ba14e5e1c3133a71d, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.043711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae4185-50d6-42f2-bcc0-1342f3c375ae admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.062323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.063769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.073412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 801/1609] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.074875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.084270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f436b5-28cc-4473-b536-52d90e9fc577 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.088978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4ebcd-3aee-475d-8287-949d08d2d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.089727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4ebcd-3aee-475d-8287-949d08d2d728 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.089997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4ebcd-3aee-475d-8287-949d08d2d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.090260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4ebcd-3aee-475d-8287-949d08d2d728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.090916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4ebcd-3aee-475d-8287-949d08d2d728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.093456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4ebcd-3aee-475d-8287-949d08d2d728 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.109286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 809/1610] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:08 2022] PUT /identity/v3/projects/b71890b7a23944ee939c4bd13408063f/users/4ce12f8401f3479ba14e5e1c3133a71d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:09.110689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02fbd4-42ef-4bf4-8d53-966e079ff6ef None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.111045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02fbd4-42ef-4bf4-8d53-966e079ff6ef None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.111385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02fbd4-42ef-4bf4-8d53-966e079ff6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.139492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0b02fbd4-42ef-4bf4-8d53-966e079ff6ef None None] MFA Rules not processed for user `18df998dc1cb47d5b5a860bbc21cf705`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.140779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0b02fbd4-42ef-4bf4-8d53-966e079ff6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.193567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 802/1611] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:09.194979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2e717-e889-4676-a6aa-7839d0461c19 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.195313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2e717-e889-4676-a6aa-7839d0461c19 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.195587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2e717-e889-4676-a6aa-7839d0461c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.217801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 810/1612] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.222232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b7b5f-1cee-48b4-ace2-4d50dc1f7c06 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.222232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b7b5f-1cee-48b4-ace2-4d50dc1f7c06 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.222232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b7b5f-1cee-48b4-ace2-4d50dc1f7c06 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.224054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 811/1613] 10.222.0.17 () {54 vars in 912 bytes} [Wed Aug 24 16:44:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:09.225433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0113173-c3b2-42e2-9a8f-a0e7ff83a08e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.225716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0113173-c3b2-42e2-9a8f-a0e7ff83a08e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.225984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0113173-c3b2-42e2-9a8f-a0e7ff83a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.241121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4ce2e717-e889-4676-a6aa-7839d0461c19 None None] MFA Rules not processed for user `1abbb93d22624eaa8a14f7533b485e81`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.243310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ce2e717-e889-4676-a6aa-7839d0461c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.248744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b0113173-c3b2-42e2-9a8f-a0e7ff83a08e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.250198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b0113173-c3b2-42e2-9a8f-a0e7ff83a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.298484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:09.316551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 803/1614] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.317915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.318201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.318468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.327506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 812/1615] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.328842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ba63-b2bc-4584-9cab-497092ed3172 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.329129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ba63-b2bc-4584-9cab-497092ed3172 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.329582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ba63-b2bc-4584-9cab-497092ed3172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.341081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.342327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.350485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 804/1616] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.351368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.353507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-59a6ba63-b2bc-4584-9cab-497092ed3172 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.355407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-59a6ba63-b2bc-4584-9cab-497092ed3172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.363366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7dfeb-6e30-4f78-84c5-532e078156a4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.365231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 813/1617] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.369389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e032b2-124b-42ad-8ec0-86877c55ec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.369553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.369856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.370216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.370354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e032b2-124b-42ad-8ec0-86877c55ec66 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.371508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e032b2-124b-42ad-8ec0-86877c55ec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.372109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e032b2-124b-42ad-8ec0-86877c55ec66 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.372881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e032b2-124b-42ad-8ec0-86877c55ec66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.375411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e032b2-124b-42ad-8ec0-86877c55ec66 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.386574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:09.392424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.393860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.465050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 805/1618] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:09.466374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51046c6-35d8-4ecf-9735-e0c0b93b9949 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.466749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51046c6-35d8-4ecf-9735-e0c0b93b9949 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.467949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51046c6-35d8-4ecf-9735-e0c0b93b9949 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.468735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 806/1619] 10.222.0.17 () {54 vars in 962 bytes} [Wed Aug 24 16:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:09.472509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 814/1620] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.473515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.473758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51046c6-35d8-4ecf-9735-e0c0b93b9949 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.533443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262aa341-591f-469a-8bf2-a3e1f2530bd6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.537010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51046c6-35d8-4ecf-9735-e0c0b93b9949 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.537394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.538079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.538302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.538568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.540805 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.541390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.541717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.541999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.548817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.550321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.551427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.552921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.559820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 815/1621] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.560653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.561929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.563055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 807/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.564047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.574757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41153443-bbf0-4533-b13d-8fe434958c7b service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.574899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3da42f-7556-44fb-8a32-59ff700f1a1d service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.578978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.580268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.580458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ba36e-f673-4200-96f0-c01a4451f865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.580596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.580596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.580777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba36e-f673-4200-96f0-c01a4451f865 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.581245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba36e-f673-4200-96f0-c01a4451f865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.581578 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba36e-f673-4200-96f0-c01a4451f865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.593758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.594340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ba36e-f673-4200-96f0-c01a4451f865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.595598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.596008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ba36e-f673-4200-96f0-c01a4451f865 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.607621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 816/1623] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.608443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 808/1624] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.608659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.609895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.610293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.610637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.619845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b025-6a3b-49da-943d-15c5db8114b0 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.640472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.641017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:09.641169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.641434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.641762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.642486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.643181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:09.644607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.656543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 809/1625] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Aug 24 16:44:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:09.658180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.666185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 817/1626] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:09.668610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.670768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252b1f0-e0b0-4e34-80f4-ab19932000a0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.675508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.676489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.676929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.677530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.694094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.694226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87479b68-9bdc-4d3a-87d2-ea766a165e50 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.695585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.705118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.705118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.705118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.705118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.720371 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 810/1627] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.721694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.735922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f0037-0f31-4849-a030-e394b29ee828 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.741642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:09.742424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:09.742704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:09.743066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] PATH_INFO: `/v3/projects/1bca1b2b245d4aee97ea46ac840af3d5/users/c9315523113f4114881dfc57c2837f20/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:09.773749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bca1b2b245d4aee97ea46ac840af3d5, user_id=c9315523113f4114881dfc57c2837f20, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.778707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.818799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:09.821011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:09.834512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 818/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:09.898284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 811/1629] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:09 2022] PUT /identity/v3/projects/1bca1b2b245d4aee97ea46ac840af3d5/users/c9315523113f4114881dfc57c2837f20/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:09.903124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:09.920306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f4c2-c82e-432d-8198-8919f44964f7 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:09.992519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adce3d-4118-436a-ba17-1831ae801fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:10.000114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adce3d-4118-436a-ba17-1831ae801fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:10.000467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adce3d-4118-436a-ba17-1831ae801fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:10.000739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adce3d-4118-436a-ba17-1831ae801fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:10.001542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adce3d-4118-436a-ba17-1831ae801fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:10.003416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adce3d-4118-436a-ba17-1831ae801fa2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:10.017787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 819/1630] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:10.022179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:10.035682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:10.040158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ae872-7bfe-48a1-84d3-c4a739e8ec7f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:10.067174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf18c7-e283-4011-8046-7635079aa619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:10.067174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf18c7-e283-4011-8046-7635079aa619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:10.067174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf18c7-e283-4011-8046-7635079aa619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:10.067174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf18c7-e283-4011-8046-7635079aa619 admin admin] PATH_INFO: `/v3/projects/1bca1b2b245d4aee97ea46ac840af3d5/users/c9315523113f4114881dfc57c2837f20/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:10.067812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf18c7-e283-4011-8046-7635079aa619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bca1b2b245d4aee97ea46ac840af3d5, user_id=c9315523113f4114881dfc57c2837f20, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:10.067812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf18c7-e283-4011-8046-7635079aa619 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:10.092421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 812/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:10 2022] PUT /identity/v3/projects/1bca1b2b245d4aee97ea46ac840af3d5/users/c9315523113f4114881dfc57c2837f20/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:10.102705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a4c0c-4b43-48cf-86f6-ed81aaff7bf2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:10.102858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a4c0c-4b43-48cf-86f6-ed81aaff7bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:10.103278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a4c0c-4b43-48cf-86f6-ed81aaff7bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:10.155014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-764a4c0c-4b43-48cf-86f6-ed81aaff7bf2 None None] MFA Rules not processed for user `c9315523113f4114881dfc57c2837f20`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:10.156998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-764a4c0c-4b43-48cf-86f6-ed81aaff7bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:10.298667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 820/1632] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:10.385741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb18eb-4c9c-46a5-8c37-0c976a6159bf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:10.386223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb18eb-4c9c-46a5-8c37-0c976a6159bf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:10.386527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb18eb-4c9c-46a5-8c37-0c976a6159bf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:10.388991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 813/1633] 10.222.0.17 () {54 vars in 912 bytes} [Wed Aug 24 16:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:10.399067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af48e4-3289-484f-a532-167dde39c807 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:10.399413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af48e4-3289-484f-a532-167dde39c807 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:10.399687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af48e4-3289-484f-a532-167dde39c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:10.427841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-75af48e4-3289-484f-a532-167dde39c807 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:10.429181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-75af48e4-3289-484f-a532-167dde39c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:10.518785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 821/1634] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:44:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:10.799796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb18eb-4c9c-46a5-8c37-0c976a6159bf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:10.862830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb18eb-4c9c-46a5-8c37-0c976a6159bf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:10.866536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:10.868229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:10.868229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:10.868229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:10.878479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:10.880184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:10.886373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:10.892192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 814/1635] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:11.061510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af48e4-3289-484f-a532-167dde39c807 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:11.078524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af48e4-3289-484f-a532-167dde39c807 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:11.087284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:11.088223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:11.088751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:11.089266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:11.114752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:11.116243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:11.144941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 822/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:44:11.657841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:11.665532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:11.667879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0d0b1-8b1f-4260-87d4-c48b88e1c7a9 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:11.667879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d84aa-ab6b-41a5-aaa9-99960b9720a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:11.667879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d84aa-ab6b-41a5-aaa9-99960b9720a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:11.667879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d84aa-ab6b-41a5-aaa9-99960b9720a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:11.670208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d84aa-ab6b-41a5-aaa9-99960b9720a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:11.670770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:11.670770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ca23f-bf18-4215-9aa6-e7bbe54674f4 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:11.670770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:11.670770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:11.671325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:11.671325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] PATH_INFO: `/v3/users/18df998dc1cb47d5b5a860bbc21cf705` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:11.671325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18df998dc1cb47d5b5a860bbc21cf705)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:11.671537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d84aa-ab6b-41a5-aaa9-99960b9720a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:11.671635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:11.671704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d84aa-ab6b-41a5-aaa9-99960b9720a6 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:11.671704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 815/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:11.955648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18df998dc1cb47d5b5a860bbc21cf705'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:11.958970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18df998dc1cb47d5b5a860bbc21cf705'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:11.971463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18df998dc1cb47d5b5a860bbc21cf705'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:12.131387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18df998dc1cb47d5b5a860bbc21cf705'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:12.133176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18df998dc1cb47d5b5a860bbc21cf705'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:12.187972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 823/1638] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:11 2022] DELETE /identity/v3/users/18df998dc1cb47d5b5a860bbc21cf705 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:12.192922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:12.193327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:12.193735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:12.241024 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:12.242603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:12.337306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 816/1639] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:12.379969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:12.386438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:12.531115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd21b1-8c8e-4e06-ab17-360e46f3a954 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:12.542310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57462047-a5c1-4c87-abb2-cf66931fc7cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:12.542310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57462047-a5c1-4c87-abb2-cf66931fc7cc service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:12.542310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57462047-a5c1-4c87-abb2-cf66931fc7cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:12.542596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57462047-a5c1-4c87-abb2-cf66931fc7cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:12.562166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57462047-a5c1-4c87-abb2-cf66931fc7cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:12.563646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57462047-a5c1-4c87-abb2-cf66931fc7cc service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:12.588251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 824/1640] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:14.181009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:14.184299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:14.217459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851d191-4b50-42bb-91c1-271eba8a1065 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:14.251052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:14.254991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:14.255339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:14.255621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] PATH_INFO: `/v3/projects/6dd83e5ccf51423e876b0dbe1b661bc5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:14.257520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd83e5ccf51423e876b0dbe1b661bc5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:14.264798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:14.451267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd83e5ccf51423e876b0dbe1b661bc5', 'request_id': 'req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:14.550010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd83e5ccf51423e876b0dbe1b661bc5', 'request_id': 'req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:14.558094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd83e5ccf51423e876b0dbe1b661bc5', 'request_id': 'req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:14.571469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] The token cache is being invalidate because project 6dd83e5ccf51423e876b0dbe1b661bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:14.572043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:14.573080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6d2dfcc1-d6a4-40fa-875d-65541e51cdb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:14.575430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 817/1641] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:14 2022] DELETE /identity/v3/projects/6dd83e5ccf51423e876b0dbe1b661bc5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:14.580307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115c39-beda-462b-b013-499acfc79195 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:14.580860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115c39-beda-462b-b013-499acfc79195 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:14.581718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9115c39-beda-462b-b013-499acfc79195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:14.607715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d9115c39-beda-462b-b013-499acfc79195 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:14.609423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d9115c39-beda-462b-b013-499acfc79195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:14.656763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:14.737782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 825/1642] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:14.754554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:14.756819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:14.758665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:14.803332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:14.806680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:14.819957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 818/1643] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:14.821596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115c39-beda-462b-b013-499acfc79195 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:14.835277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9115c39-beda-462b-b013-499acfc79195 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:14.844579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:14.845220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:14.845498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:14.846131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:14.846807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:14.847111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:14.849341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:14.871797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 826/1644] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:14.873021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:14.884597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b480ab-1484-4994-a4d9-024eb973b252 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:14.892243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:14.892905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:14.893179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:14.893496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:14.897756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:14.897756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:14.931951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:15.704007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 819/1645] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:14 2022] POST /identity/v3/projects => generated 369 bytes in 829 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:15.708594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:15.723350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b865c9-c656-458a-9401-84625fb3b78b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:15.929125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:15.931707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:15.935020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:15.935669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:15.936119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:15.936443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:15.937152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:15.939276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:15.965381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae388d50-4b6b-402e-be74-dc0423c973ff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:15.987643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:15.988175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:15.989792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:15.992031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] PATH_INFO: `/v3/users/fc7edc3d8ae84e0585965afe4486bb27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:15.995877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7edc3d8ae84e0585965afe4486bb27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:15.997402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:15.997402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Aug 24 16:44:15.997402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: Aug 24 16:44:16.006965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:16.100283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 827/1646] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:15 2022] POST /identity/v3/users => generated 285 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:16.108035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:16.147375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb6d7-5794-4b93-ae05-a0f0834edc6e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:16.169183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:16.169828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:16.170094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:16.171122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:16.171995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:16.174029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:16.222383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 828/1647] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:16.234411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:16.253278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84989477-dc41-4bbd-8753-9d7f053e9d09 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:16.260152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:16.261915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:16.262620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:16.263058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:16.263939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:16.265901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:16.288105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 829/1648] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:16.292132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:16.307970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f7ccc-af63-4e2e-979a-3f34d332345e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:16.317732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be7a8-9672-47ae-b6b1-feed8021624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:16.318714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be7a8-9672-47ae-b6b1-feed8021624f admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:16.319197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be7a8-9672-47ae-b6b1-feed8021624f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:16.320558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be7a8-9672-47ae-b6b1-feed8021624f admin admin] PATH_INFO: `/v3/projects/4a4e50fb9544418f9636f3ea20d1c949/users/891abf7a88014845865a7283dc7af94f/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:16.347455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9be7a8-9672-47ae-b6b1-feed8021624f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e50fb9544418f9636f3ea20d1c949, user_id=891abf7a88014845865a7283dc7af94f, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:16.355669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9be7a8-9672-47ae-b6b1-feed8021624f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:16.370911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7edc3d8ae84e0585965afe4486bb27'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:16.496691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7edc3d8ae84e0585965afe4486bb27'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:16.499691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7edc3d8ae84e0585965afe4486bb27'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:16.520760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 830/1649] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:16 2022] PUT /identity/v3/projects/4a4e50fb9544418f9636f3ea20d1c949/users/891abf7a88014845865a7283dc7af94f/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:16.532011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:16.532715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:16.533446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:16.554365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7edc3d8ae84e0585965afe4486bb27'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:16.597470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] MFA Rules not processed for user `891abf7a88014845865a7283dc7af94f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:16.599014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:16.600068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd7d472e-933a-4cc4-8c4b-77879f168727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7edc3d8ae84e0585965afe4486bb27'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:16.612799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 820/1650] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:15 2022] DELETE /identity/v3/users/fc7edc3d8ae84e0585965afe4486bb27 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:16.616933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:16.617215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:16.617488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:16.659512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:16.664132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:16.766430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 831/1651] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:16.773372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:16.775348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:16.822076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 821/1652] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:16.974004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f614ea-9105-4239-812b-7db3ad3c9fcf None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:16.982899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:16.990690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:16.995545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:16.997413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:16.997785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:16.998075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:16.998726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:17.000762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:17.103124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345d21e-cce6-4da0-88a3-820c8630aedc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:17.109964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:17.109964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:17.109964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:17.109964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:17.125056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:17.127526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:17.149175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 832/1653] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:16 2022] POST /identity/v3/projects => generated 369 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:17.149175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:17.178745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 822/1654] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:17.237568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:17.412467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a8138-a117-4039-9fef-460903eb07dd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:17.453416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:17.454331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:17.455837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:17.456274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:17.458303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:17.461384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:17.606483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 833/1655] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:17 2022] POST /identity/v3/users => generated 291 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:17.610640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:17.624218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf5ca-d163-4f91-9bf2-033c896b1d08 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:17.630947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:17.638999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:17.638999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:17.638999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:17.638999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:17.638999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:17.638999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:17.656128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:17.686572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 823/1656] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:17.689531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bde86-fba3-48c5-abde-d60531175c51 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:17.705331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:17.707663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:17.708200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:17.710016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] PATH_INFO: `/v3/projects/fcc917128f3642828c7e2af44274e383/users/9603b1f64ecd40049f4a1603ea715863/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:17.735773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc917128f3642828c7e2af44274e383, user_id=9603b1f64ecd40049f4a1603ea715863, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:17.741958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:17.887922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:17.895297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 834/1657] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:17 2022] PUT /identity/v3/projects/fcc917128f3642828c7e2af44274e383/users/9603b1f64ecd40049f4a1603ea715863/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:17.904938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93730593-dc83-4d0c-a3bf-410043c77a7a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:18.055678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:18.058413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:18.059027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:18.059470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:18.060521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:18.066525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:18.093220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 824/1658] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:18.098134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:18.116712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429d59b-1b79-4cdc-809b-3b555411a189 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:18.131182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d38a5-1ce2-4344-8091-479e772a25c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:18.132173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d38a5-1ce2-4344-8091-479e772a25c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:18.134109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d38a5-1ce2-4344-8091-479e772a25c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:18.136475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d38a5-1ce2-4344-8091-479e772a25c3 admin admin] PATH_INFO: `/v3/projects/fcc917128f3642828c7e2af44274e383/users/9603b1f64ecd40049f4a1603ea715863/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:18.144820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:18.154476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:18.154725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d38a5-1ce2-4344-8091-479e772a25c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc917128f3642828c7e2af44274e383, user_id=9603b1f64ecd40049f4a1603ea715863, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:18.162207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d38a5-1ce2-4344-8091-479e772a25c3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:18.177087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdb71c-4cde-4b04-a7e9-3fcf2af3fd27 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:18.210613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 835/1659] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:18 2022] PUT /identity/v3/projects/fcc917128f3642828c7e2af44274e383/users/9603b1f64ecd40049f4a1603ea715863/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:18.211652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:18.211652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:18.211652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:18.211652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] PATH_INFO: `/v3/users/7cc997ec4a3f460197d19cedb78cfee8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:18.212073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc997ec4a3f460197d19cedb78cfee8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:18.216524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:18.224209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:18.224565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:18.224848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:18.303690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] MFA Rules not processed for user `9603b1f64ecd40049f4a1603ea715863`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:18.308291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:18.455506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 836/1660] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:18.468262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:18.472197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:18.481558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc997ec4a3f460197d19cedb78cfee8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:18.572174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc997ec4a3f460197d19cedb78cfee8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:18.573752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc997ec4a3f460197d19cedb78cfee8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:18.599058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc997ec4a3f460197d19cedb78cfee8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:18.621510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5af46-6f19-4219-8173-1379aae68822 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:18.629248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:18.629807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:18.630091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:18.630364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:18.649567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:18.652069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c0de49dc-5c66-4470-95d8-a91e8fc5f5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc997ec4a3f460197d19cedb78cfee8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:18.652792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:18.664907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 825/1661] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:18 2022] DELETE /identity/v3/users/7cc997ec4a3f460197d19cedb78cfee8 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:18.670235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:18.670755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:18.671852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:18.687189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 837/1662] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:18.724060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:18.730023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:18.748857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:18.844511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 826/1663] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:18.961329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:19.076448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2f109-5e61-4793-9b1d-ca3adcc95303 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:19.196995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18c847-55a5-479d-a22d-9878eff2714f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:19.202636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18c847-55a5-479d-a22d-9878eff2714f service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:19.202636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18c847-55a5-479d-a22d-9878eff2714f service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:19.202636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18c847-55a5-479d-a22d-9878eff2714f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:19.218955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18c847-55a5-479d-a22d-9878eff2714f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:19.220548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18c847-55a5-479d-a22d-9878eff2714f service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:19.270722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 838/1664] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:20.698986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:20.700777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:20.734988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b29290-6d10-4519-a544-b08c777fbb8d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:20.747459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:20.750589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:20.751211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:20.752470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] PATH_INFO: `/v3/projects/f9652b4a38de4d4087e33ff15f193205` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:20.754553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9652b4a38de4d4087e33ff15f193205)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:20.763024 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:20.765059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:20.952291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9652b4a38de4d4087e33ff15f193205', 'request_id': 'req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:21.080222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9652b4a38de4d4087e33ff15f193205', 'request_id': 'req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:21.092774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9652b4a38de4d4087e33ff15f193205', 'request_id': 'req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:21.130530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] The token cache is being invalidate because project f9652b4a38de4d4087e33ff15f193205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:21.131047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:21.136081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-373d0b4a-7e36-49b2-aad7-3cc34a0df33c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:21.138676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 827/1665] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:20 2022] DELETE /identity/v3/projects/f9652b4a38de4d4087e33ff15f193205 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:21.143506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.143978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.144318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.191806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:21.193124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.273310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:21.497909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 839/1666] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:21.499202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.499489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.499765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.558360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:21.559605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.594258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 828/1667] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:21.599106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:21.615231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c97c9-eba8-4aa7-a32e-9c19bd01a019 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:21.627180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:21.628982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.629330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.631420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.632259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:21.637720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:21.650730 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c60969-6356-4c39-b148-f2bc49238f4d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:21.690733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:21.693824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 840/1668] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:21.706061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1159d09-59f4-40c8-bd23-99b8b32279dc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:21.720606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:21.721260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.721534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.721806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.722618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:21.725059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:21.805348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378fac5-c135-44bb-9db1-c0ad9145b32f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:21.805640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378fac5-c135-44bb-9db1-c0ad9145b32f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:21.805916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378fac5-c135-44bb-9db1-c0ad9145b32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:21.883259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8378fac5-c135-44bb-9db1-c0ad9145b32f None None] MFA Rules not processed for user `491e2b9a44bd47b190b1a0b1250bf1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:21.884589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8378fac5-c135-44bb-9db1-c0ad9145b32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:21.891016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 829/1669] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:21.893975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.061013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71c93a-ca62-443e-8b33-d0a44ef0476a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.074598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 841/1670] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:22.081939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.082590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.082990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.083283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.085555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.086023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.086390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.086839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.105397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.148774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] MFA Rules not processed for user `491e2b9a44bd47b190b1a0b1250bf1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:22.150031 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.236440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 830/1671] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:21 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:22.240993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.255023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 842/1672] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:22.262829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5e6b1-14c0-40e4-96f7-2ece3f35d20c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.266248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.273139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.290086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.290703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.290984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.295835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.296597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.305723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.349837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 831/1673] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:22.355558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.387112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e4c50-b983-4337-89dc-6e0acce1c6af admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.401972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.402613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.402895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.403157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.403822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.405492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.418419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:22.430685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 832/1674] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:22.441771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.482542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb02987-d900-4acc-840e-c0d5cec12bf6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.507906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073437c6-6aae-4e4f-9405-d81ae7a9fc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.512051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073437c6-6aae-4e4f-9405-d81ae7a9fc52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.512372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073437c6-6aae-4e4f-9405-d81ae7a9fc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.515104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073437c6-6aae-4e4f-9405-d81ae7a9fc52 admin admin] PATH_INFO: `/v3/projects/356c77a6ac98435f8aca0b1b61def246/users/a30d975e84924d09b05165d605f95d8d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.520283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea9130-01e2-4893-87b0-f783a0462032 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.536953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.537655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.537981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.538111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.555320 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073437c6-6aae-4e4f-9405-d81ae7a9fc52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356c77a6ac98435f8aca0b1b61def246, user_id=a30d975e84924d09b05165d605f95d8d, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.558936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073437c6-6aae-4e4f-9405-d81ae7a9fc52 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.564512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.566124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.587569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 843/1675] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:22.588722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.590190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.611924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405b29-7f24-40d8-9376-aa89df7ee874 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:22.620123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:22.620562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.621001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.621276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] PATH_INFO: `/v3/projects/2ac68a42d7514dc8af4c0cdc1d423f8e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.622523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac68a42d7514dc8af4c0cdc1d423f8e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:22.623860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 16:44:22.623860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 24 16:44:22.623860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: Aug 24 16:44:22.625267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:22.664945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 833/1676] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:22 2022] PUT /identity/v3/projects/356c77a6ac98435f8aca0b1b61def246/users/a30d975e84924d09b05165d605f95d8d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:22.676582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:22.676869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:22.677154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:22.727497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] MFA Rules not processed for user `a30d975e84924d09b05165d605f95d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:22.729200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:22.849967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 834/1677] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:22.861279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:22.904799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac68a42d7514dc8af4c0cdc1d423f8e', 'request_id': 'req-c57189ac-8e28-4abd-8d43-e4deb9f738a6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:22.986341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac68a42d7514dc8af4c0cdc1d423f8e', 'request_id': 'req-c57189ac-8e28-4abd-8d43-e4deb9f738a6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.097107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac68a42d7514dc8af4c0cdc1d423f8e', 'request_id': 'req-c57189ac-8e28-4abd-8d43-e4deb9f738a6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.097453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e316b1-ec80-49b0-af4e-aacefd55fa50 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:23.118744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] The token cache is being invalidate because project 2ac68a42d7514dc8af4c0cdc1d423f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:23.120458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.121456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.130482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 844/1678] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:22 2022] DELETE /identity/v3/projects/2ac68a42d7514dc8af4c0cdc1d423f8e => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:23.134720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:23.139044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:23.147147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:23.236840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:23.237746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.238025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.238361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.239095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:23.243247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:23.309674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57189ac-8e28-4abd-8d43-e4deb9f738a6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:23.316521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:23.317442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.317731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.318340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] PATH_INFO: `/v3/users/c9315523113f4114881dfc57c2837f20` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.320561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9315523113f4114881dfc57c2837f20)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:23.325252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:23.380767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 835/1679] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:22 2022] POST /identity/v3/projects => generated 363 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:23.387850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:23.394437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:23.566674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebac5a4-c28e-4521-ab55-23959f185a74 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:23.577961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518edd00-d830-43e1-a97b-71e5899552bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:23.579328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518edd00-d830-43e1-a97b-71e5899552bd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.579617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518edd00-d830-43e1-a97b-71e5899552bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.579897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518edd00-d830-43e1-a97b-71e5899552bd admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.580927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518edd00-d830-43e1-a97b-71e5899552bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:23.583409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518edd00-d830-43e1-a97b-71e5899552bd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:23.676219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9315523113f4114881dfc57c2837f20'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.677307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9315523113f4114881dfc57c2837f20'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.702374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:23.709067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9315523113f4114881dfc57c2837f20'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.720121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 836/1680] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:23 2022] POST /identity/v3/users => generated 288 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:23.728375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f9bd0-2f92-4fa5-8365-d617adb77030 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.728375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f9bd0-2f92-4fa5-8365-d617adb77030 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.728375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f9bd0-2f92-4fa5-8365-d617adb77030 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.728375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 837/1681] 10.222.0.17 () {54 vars in 912 bytes} [Wed Aug 24 16:44:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:23.730975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f9bd0-2f92-4fa5-8365-d617adb77030 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:23.735710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9b4f9bd0-2f92-4fa5-8365-d617adb77030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:23.816236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f9bd0-2f92-4fa5-8365-d617adb77030 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:23.823407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9315523113f4114881dfc57c2837f20'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.824245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9315523113f4114881dfc57c2837f20'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:23.825404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:23.826543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:23.830714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:23.830714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] PATH_INFO: `/v3/users/891abf7a88014845865a7283dc7af94f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:23.834962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] RBAC: Authorizing `identity:delete_user(user_id=891abf7a88014845865a7283dc7af94f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:23.855412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:23.909760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 845/1682] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:23 2022] DELETE /identity/v3/users/c9315523113f4114881dfc57c2837f20 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:23.913087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:23.923807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:24.081618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891abf7a88014845865a7283dc7af94f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:24.159508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba903307-98e1-43ab-8008-b45a8b4e4316 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:24.170816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad3077-5950-46ae-afda-b58c945f8e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:24.171760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad3077-5950-46ae-afda-b58c945f8e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.172243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad3077-5950-46ae-afda-b58c945f8e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.172696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad3077-5950-46ae-afda-b58c945f8e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:24.173519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad3077-5950-46ae-afda-b58c945f8e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:24.176982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891abf7a88014845865a7283dc7af94f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:24.177830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891abf7a88014845865a7283dc7af94f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:24.188233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891abf7a88014845865a7283dc7af94f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:24.190001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad3077-5950-46ae-afda-b58c945f8e9b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:24.218854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 846/1683] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:24.221369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891abf7a88014845865a7283dc7af94f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:24.222139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9112d9-944a-4c0a-97f2-f5f8fe4465c6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.222581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9112d9-944a-4c0a-97f2-f5f8fe4465c6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.224292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9112d9-944a-4c0a-97f2-f5f8fe4465c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:24.245981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 838/1684] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:23 2022] DELETE /identity/v3/users/891abf7a88014845865a7283dc7af94f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:24.253847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:24.262267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:24.442767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9a9112d9-944a-4c0a-97f2-f5f8fe4465c6 None None] MFA Rules not processed for user `9bde9ceb3c7c46298626580b62e0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:24.446182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9a9112d9-944a-4c0a-97f2-f5f8fe4465c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:24.519827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9eea1-40da-4781-aba7-36615f6a829f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:24.545987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:24.546810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.547362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.547757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] PATH_INFO: `/v3/projects/b90498cf7cca4d938c8cdb4e3bf4b015/users/88e1847f005b4ff6bd2dc3e7846bc130/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:24.603415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 847/1685] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:24.605374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.605662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.605930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:24.606561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90498cf7cca4d938c8cdb4e3bf4b015, user_id=88e1847f005b4ff6bd2dc3e7846bc130, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:24.609637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:24.649686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:24.653644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:24.678593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 848/1686] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:24.679497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:24.683433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:24.711627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaea35-5102-49eb-9005-25770edc7aad None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:24.725940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b379ef7-8f2e-454a-b630-6b5125ca18ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:24.726625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379ef7-8f2e-454a-b630-6b5125ca18ad service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.726931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379ef7-8f2e-454a-b630-6b5125ca18ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.727199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b379ef7-8f2e-454a-b630-6b5125ca18ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:24.728665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 839/1687] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:24 2022] PUT /identity/v3/projects/b90498cf7cca4d938c8cdb4e3bf4b015/users/88e1847f005b4ff6bd2dc3e7846bc130/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:24.730427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:24.747821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b379ef7-8f2e-454a-b630-6b5125ca18ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:24.749770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b379ef7-8f2e-454a-b630-6b5125ca18ad service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:24.773302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 849/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:24.776646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.776933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.777657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:24.849767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] MFA Rules not processed for user `1d9f7755863d45bfb04a1498bd62e367`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:24.856504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:24.901129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec70d-940a-4152-841f-e143ad565019 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:24.911792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:24.917205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:24.917205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:24.917205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.005695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 850/1689] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:25.009978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.031903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.033785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:25.039120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e32b2-8ae2-4a38-8ff2-d6b47dd7323d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:25.051967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:25.052838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.055336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.055831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.057481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.062910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 840/1690] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:25.064230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:25.067525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.072235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:25.099666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 851/1691] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:25.101515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.126649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6c99b-8343-4698-9cd0-d504419adf97 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:25.144384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7015bfc-f327-424d-ab6a-c959d230636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:25.145413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015bfc-f327-424d-ab6a-c959d230636b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.145697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015bfc-f327-424d-ab6a-c959d230636b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.145971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015bfc-f327-424d-ab6a-c959d230636b admin admin] PATH_INFO: `/v3/projects/b90498cf7cca4d938c8cdb4e3bf4b015/users/88e1847f005b4ff6bd2dc3e7846bc130/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.152186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015bfc-f327-424d-ab6a-c959d230636b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90498cf7cca4d938c8cdb4e3bf4b015, user_id=88e1847f005b4ff6bd2dc3e7846bc130, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.163041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015bfc-f327-424d-ab6a-c959d230636b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:25.190752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:25.216607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 852/1692] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:25 2022] PUT /identity/v3/projects/b90498cf7cca4d938c8cdb4e3bf4b015/users/88e1847f005b4ff6bd2dc3e7846bc130/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:25.218172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.218619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.219061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.263767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] MFA Rules not processed for user `88e1847f005b4ff6bd2dc3e7846bc130`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:25.265324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:25.295494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d6d5e-1edb-464e-9d60-392ce9a9eb9a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:25.372499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 853/1693] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:25.378443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.381452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:25.459368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:25.460277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.464649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.464649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.595510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1fc20-247d-4999-80f1-de3325e91057 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:25.618848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:25.620010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.620264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.620567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.651885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.660308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:25.681397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.685525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:25.697315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 854/1694] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:44:25.698151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.707230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:25.708393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 841/1695] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:25.710164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:25.713506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:25.823051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:25.924188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70175d5f-1d64-41aa-8360-4ae0754d44c9 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:25.930654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:25.930654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:25.930654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:25.930654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] PATH_INFO: `/v3/projects/1bca1b2b245d4aee97ea46ac840af3d5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:25.940182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bca1b2b245d4aee97ea46ac840af3d5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:25.950035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:25.990209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71c330-50b3-46ed-932e-6347ad09f412 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:26.008334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:26.012990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:26.014831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:26.015510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] PATH_INFO: `/v3/users/1abbb93d22624eaa8a14f7533b485e81` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:26.020602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abbb93d22624eaa8a14f7533b485e81)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:26.033230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:26.461788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bca1b2b245d4aee97ea46ac840af3d5', 'request_id': 'req-4130ffdf-fcc9-41b6-81f7-37826497bbd3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.499026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abbb93d22624eaa8a14f7533b485e81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.501225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abbb93d22624eaa8a14f7533b485e81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.526103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abbb93d22624eaa8a14f7533b485e81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.548556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bca1b2b245d4aee97ea46ac840af3d5', 'request_id': 'req-4130ffdf-fcc9-41b6-81f7-37826497bbd3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.559991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bca1b2b245d4aee97ea46ac840af3d5', 'request_id': 'req-4130ffdf-fcc9-41b6-81f7-37826497bbd3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.616070 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] The token cache is being invalidate because project 1bca1b2b245d4aee97ea46ac840af3d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:26.616554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.617369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.619846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 842/1696] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:25 2022] DELETE /identity/v3/projects/1bca1b2b245d4aee97ea46ac840af3d5 => generated 0 bytes in 910 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:26.621567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:26.624967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:26.658608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abbb93d22624eaa8a14f7533b485e81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.660103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-737fef76-d559-4319-b818-87ac82b35c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abbb93d22624eaa8a14f7533b485e81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:26.713090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 855/1697] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:25 2022] DELETE /identity/v3/users/1abbb93d22624eaa8a14f7533b485e81 => generated 0 bytes in 1015 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:26.715130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214602c-7edc-41a2-873d-b4b53adc48ab None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:26.715419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214602c-7edc-41a2-873d-b4b53adc48ab None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:26.715695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214602c-7edc-41a2-873d-b4b53adc48ab None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:26.726414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 856/1698] 10.222.0.17 () {54 vars in 961 bytes} [Wed Aug 24 16:44:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:26.727321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214602c-7edc-41a2-873d-b4b53adc48ab None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:26.734818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b214602c-7edc-41a2-873d-b4b53adc48ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:26.801687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130ffdf-fcc9-41b6-81f7-37826497bbd3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:26.873421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:26.874451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:26.874878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:26.875593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:26.878027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:26.889946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214602c-7edc-41a2-873d-b4b53adc48ab None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:26.909834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:26.910275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:26.910923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:26.910923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] PATH_INFO: `/v3/projects/4a4e50fb9544418f9636f3ea20d1c949` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:26.912054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4e50fb9544418f9636f3ea20d1c949)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:26.914382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:26.975126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:26.977772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76882291-067a-44a9-87e5-ec454a8cdc97 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:26.990406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 843/1699] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:26.999992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:27.007158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9da3-78b1-4e10-922e-9358ec029bef None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.008778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9da3-78b1-4e10-922e-9358ec029bef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.008778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9da3-78b1-4e10-922e-9358ec029bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.070469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1b4b9da3-78b1-4e10-922e-9358ec029bef None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:27.075027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1b4b9da3-78b1-4e10-922e-9358ec029bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.086872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4e50fb9544418f9636f3ea20d1c949', 'request_id': 'req-d9ac3bc3-5005-4503-8429-5e2f2367d798'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.097414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4e50fb9544418f9636f3ea20d1c949', 'request_id': 'req-d9ac3bc3-5005-4503-8429-5e2f2367d798'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.176295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4e50fb9544418f9636f3ea20d1c949', 'request_id': 'req-d9ac3bc3-5005-4503-8429-5e2f2367d798'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.195133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] The token cache is being invalidate because project 4a4e50fb9544418f9636f3ea20d1c949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:27.195787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.196575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 844/1700] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:27.198508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9ac3bc3-5005-4503-8429-5e2f2367d798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.199524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b3e87-0b6a-44d3-8222-59f8373f72b6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.204104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b3e87-0b6a-44d3-8222-59f8373f72b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.205078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 857/1701] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:26 2022] DELETE /identity/v3/projects/4a4e50fb9544418f9636f3ea20d1c949 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:27.205513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b3e87-0b6a-44d3-8222-59f8373f72b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.210784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e314-b078-41ad-9d7d-478b9ba16338 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.211602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e314-b078-41ad-9d7d-478b9ba16338 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.212093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212e314-b078-41ad-9d7d-478b9ba16338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.263141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-394b3e87-0b6a-44d3-8222-59f8373f72b6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:27.266146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-394b3e87-0b6a-44d3-8222-59f8373f72b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.271609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e212e314-b078-41ad-9d7d-478b9ba16338 None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:27.273454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e212e314-b078-41ad-9d7d-478b9ba16338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.287003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 845/1702] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:27.288814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.298648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.299492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.379037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] MFA Rules not processed for user `7be50b041b5947d5909ed6ce516ec8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:27.380445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.437490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 858/1703] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Aug 24 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:27.438816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.439399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.439399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.505933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:27.508124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.535045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 859/1704] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:27.536146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:27.538984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.560124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e8dc9-367d-4a2f-abd2-ecd854346d70 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:27.575950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:27.586072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.586760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.587198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] PATH_INFO: `/v3/users/9603b1f64ecd40049f4a1603ea715863` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.588931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9603b1f64ecd40049f4a1603ea715863)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:27.591476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:27.654804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 846/1705] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:27.659589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:27.661355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.777461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489f78-0e36-4843-924a-e1ab5e5f50f5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:27.797077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:27.797790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.801309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.801309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.834444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:27.839869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:27.858041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 847/1706] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:27.871353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:27.894711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9603b1f64ecd40049f4a1603ea715863'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.895677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9603b1f64ecd40049f4a1603ea715863'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.912271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e5f81-95bc-4374-9fb0-16d182ff5b9a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:27.913136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9603b1f64ecd40049f4a1603ea715863'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:27.925389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:27.926106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:27.926792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:27.926905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:27.929020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:27.945668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:27.954239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:27.976008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 848/1707] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:27.979316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:28.003947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49c562-d3d2-4ecf-80f7-4feaf070653f service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:28.011589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9603b1f64ecd40049f4a1603ea715863'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:28.012403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bafb5c3-c1fc-4198-a421-03c59e35185e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9603b1f64ecd40049f4a1603ea715863'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:28.016732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:28.020254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:28.020721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:28.021901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:28.024534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:28.025297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:28.033197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:28.058299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 860/1708] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:27 2022] DELETE /identity/v3/users/9603b1f64ecd40049f4a1603ea715863 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:28.064763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:28.071888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:28.073060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:28.083513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 849/1709] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:28.087702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:28.101791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:28.141395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] MFA Rules not processed for user `7be50b041b5947d5909ed6ce516ec8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:28.144512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e8a08-d89e-446a-bb8f-71604ad0ad86 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:28.147720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:28.149703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:28.278437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 861/1710] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:28.288116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:28.289915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:28.307625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:28.308656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:28.309263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:28.309723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:28.311078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:28.314008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:28.394927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 850/1711] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:28 2022] POST /identity/v3/projects => generated 401 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:28.399307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:28.513329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca71cd1-6020-4791-86f1-8559c23c7adb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:28.590896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4338c-ee5c-4ec8-aba3-d7f49ef9be0c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:28.605141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:28.638522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:28.638522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:28.638522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:28.638522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:28.638522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:28.699120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:28.699880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:28.700106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:28.718847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:28.731841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:28.833244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 851/1712] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:28 2022] POST /identity/v3/users => generated 301 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:28.840071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:28.886495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6010a0-4327-407f-bc18-422dc224a97d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:28.901425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:28.902318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:28.902884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:28.903253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:28.912084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:28.918463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:28.955556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 852/1713] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:28.959207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:28.995774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2696c19-bac1-4e86-9d83-0b164fcdb72a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:29.017391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:29.022951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.023833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.027576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.028615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.031865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.033294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661901b-4952-4648-b3d4-2627ff297982 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.035317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.079905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 853/1714] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:44:29.089501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:29.105754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd822b-4c21-4486-9d18-fcd77eda0a46 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:29.116023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 862/1715] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 828 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:29.127338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:29.128813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.129205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.129631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] PATH_INFO: `/v3/projects/d5a9718e3f7d4d96b14b2fdb21b1e324/users/644b5471a1424632933948e49a381ee2/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.163905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a9718e3f7d4d96b14b2fdb21b1e324, user_id=644b5471a1424632933948e49a381ee2, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.167871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.272596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 854/1716] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:29 2022] PUT /identity/v3/projects/d5a9718e3f7d4d96b14b2fdb21b1e324/users/644b5471a1424632933948e49a381ee2/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:29.283716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.284027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.284377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.339875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] MFA Rules not processed for user `644b5471a1424632933948e49a381ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:29.341190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:29.445778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:29.453906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 863/1717] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:29 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:29.458576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:29.570416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4ae44-05a2-41f1-9aa4-29e75bb25cd6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:29.573500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:29.575900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:29.576020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:29.576593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.576867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.577132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.577751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.580133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.605243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15713679-9093-4b38-9578-3a36b49a8abe None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:29.621495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:29.622340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.622723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.623766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] PATH_INFO: `/v3/projects/962fd0e297b24ea2a77441d8d8928d5b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.626027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=962fd0e297b24ea2a77441d8d8928d5b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.630719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.638700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:29.683240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 855/1718] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:29 2022] POST /identity/v3/projects => generated 399 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:29.684195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:29.838394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fe237-df16-4261-b110-8aab3e4a9dd5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:29.838633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962fd0e297b24ea2a77441d8d8928d5b', 'request_id': 'req-39316449-2221-4ed9-bc09-0a89379e29ca'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:29.855120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962fd0e297b24ea2a77441d8d8928d5b', 'request_id': 'req-39316449-2221-4ed9-bc09-0a89379e29ca'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:29.965421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962fd0e297b24ea2a77441d8d8928d5b', 'request_id': 'req-39316449-2221-4ed9-bc09-0a89379e29ca'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:29.969445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:29.969445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:29.969445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:29.978159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:29.979253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:29.986358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:29.994529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] The token cache is being invalidate because project 962fd0e297b24ea2a77441d8d8928d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:29.998062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:29.998062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39316449-2221-4ed9-bc09-0a89379e29ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:30.008022 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 864/1719] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:29 2022] DELETE /identity/v3/projects/962fd0e297b24ea2a77441d8d8928d5b => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:30.009840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493487f9-02ed-4e03-a98b-d88400bc591a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.011493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493487f9-02ed-4e03-a98b-d88400bc591a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.013711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493487f9-02ed-4e03-a98b-d88400bc591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.107604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-493487f9-02ed-4e03-a98b-d88400bc591a None None] MFA Rules not processed for user `4ce12f8401f3479ba14e5e1c3133a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:30.113615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-493487f9-02ed-4e03-a98b-d88400bc591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.159033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 856/1720] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:29 2022] POST /identity/v3/users => generated 306 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:30.162725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:30.166385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.201879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4814640-5680-46a9-b755-a5ae26da54db admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:30.207817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:30.208572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.208900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.209303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] PATH_INFO: `/v3/projects/fcc917128f3642828c7e2af44274e383` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.211859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc917128f3642828c7e2af44274e383)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:30.214808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:30.252842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:30.288287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 865/1721] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:30.291254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.291623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.292529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.324950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:30.324950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.333168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 866/1722] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:30.334123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:30.340463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.352980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794368d-6350-4df8-b0e7-86f3e062cbd9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:30.358564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:30.359575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.359575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.359931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.360799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:30.362571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:30.374554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 867/1723] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:30.375384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:30.377111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.405395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc917128f3642828c7e2af44274e383', 'request_id': 'req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:30.463062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043de1-d0cd-4c40-a5c8-810a806536f4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:30.496571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc917128f3642828c7e2af44274e383', 'request_id': 'req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:30.504905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc917128f3642828c7e2af44274e383', 'request_id': 'req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:30.525531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] The token cache is being invalidate because project fcc917128f3642828c7e2af44274e383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:30.525531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:30.527035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c65bf3d2-7ae9-4be8-8b0b-9dd7c5fca964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:30.539283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 857/1724] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:30 2022] DELETE /identity/v3/projects/fcc917128f3642828c7e2af44274e383 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:30.541348 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.541754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.542057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.629291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:30.632096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.639911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:30.640636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.640913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.641188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.645189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.803774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 858/1725] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:30.803774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:30.809358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:30.830600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:30.832224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbf532-8a15-4ece-a8c4-9a52e125b7d5 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:30.834534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f938d-b1ca-4662-8a73-fd74bccd3028 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:30.843913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:30.845515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.846027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.846500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] PATH_INFO: `/v3/projects/50aec55da7c24c7d92e1b82be498b993/users/780615da99fa4b299f0eff600fec6c6c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.861583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 868/1726] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:30.872853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50aec55da7c24c7d92e1b82be498b993, user_id=780615da99fa4b299f0eff600fec6c6c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:30.876254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:30.876752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:30.877134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:30.877479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:30.968119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 859/1727] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:30 2022] PUT /identity/v3/projects/50aec55da7c24c7d92e1b82be498b993/users/780615da99fa4b299f0eff600fec6c6c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:30.968119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:30.996646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:31.004802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:31.210118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 869/1728] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:31.212180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:31.249357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388caee-8e39-4c18-8e3f-dcc571852860 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:31.267951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ae985-0f40-4541-b553-022a2e64fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:31.272254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ae985-0f40-4541-b553-022a2e64fb75 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.273931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ae985-0f40-4541-b553-022a2e64fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.276097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ae985-0f40-4541-b553-022a2e64fb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.278107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ae985-0f40-4541-b553-022a2e64fb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:31.281720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd7f71-fd3c-4a8e-8d22-795b1e6f583e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:31.284900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ae985-0f40-4541-b553-022a2e64fb75 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:31.338934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 870/1729] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:31.342269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.342752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:31.343391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.343673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.343946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.344267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.345389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.347914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:31.348266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:31.349966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:31.402342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 860/1730] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:31.419781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:31.424501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:31.425853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:31.455653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 871/1731] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:31.456601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:31.468082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6086c12-6869-4502-ba19-3bdd8a858b45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:31.480715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc6851-e93c-4b4a-ba8d-89f343613656 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:31.490089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22f385-1013-4f1d-b90c-ce6425cfdc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:31.492937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22f385-1013-4f1d-b90c-ce6425cfdc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.494252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:31.495133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.495558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.495995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.496976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:31.500038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22f385-1013-4f1d-b90c-ce6425cfdc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.500261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:31.500451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22f385-1013-4f1d-b90c-ce6425cfdc54 admin admin] PATH_INFO: `/v3/projects/50aec55da7c24c7d92e1b82be498b993/users/780615da99fa4b299f0eff600fec6c6c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.506335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22f385-1013-4f1d-b90c-ce6425cfdc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50aec55da7c24c7d92e1b82be498b993, user_id=780615da99fa4b299f0eff600fec6c6c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:31.509588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22f385-1013-4f1d-b90c-ce6425cfdc54 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:31.568865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 861/1732] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:31 2022] PUT /identity/v3/projects/50aec55da7c24c7d92e1b82be498b993/users/780615da99fa4b299f0eff600fec6c6c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:31.570170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474f6e-0a1c-49b6-a35b-3a0d07b3dd8c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.570521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474f6e-0a1c-49b6-a35b-3a0d07b3dd8c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.570711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd474f6e-0a1c-49b6-a35b-3a0d07b3dd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.643025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cd474f6e-0a1c-49b6-a35b-3a0d07b3dd8c None None] MFA Rules not processed for user `7be50b041b5947d5909ed6ce516ec8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:31.644941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cd474f6e-0a1c-49b6-a35b-3a0d07b3dd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:31.654063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 872/1733] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:31 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:31.658639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:31.748665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 862/1734] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:31.748665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.748665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.748665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.833288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] MFA Rules not processed for user `780615da99fa4b299f0eff600fec6c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:31.835625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:31.909309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a112a-ffe1-4256-ac21-c2521f2e5f70 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:31.922112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:31.924311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:31.924311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:31.924617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:31.925283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:31.925591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:31.946139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:31.947695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 863/1735] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:31.948827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:31.989721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494caaf-9113-4429-966f-a3c1a44b518b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.004486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 873/1736] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:32.004486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.008795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:32.012922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.013747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.014566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.015025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.015814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.018381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.101923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc989fe4-29b7-4dfb-8dd0-dfe851980d5f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.107532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.108317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.108633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.108902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.123875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.126223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.169742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 874/1737] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:32.172950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.185424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 864/1738] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:31 2022] POST /identity/v3/users => generated 291 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:32.187115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.200383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ba086-4753-4f56-88bc-8d345bbfffe9 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.216287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56abaa-4b84-4594-a624-660c12eb28c3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.219903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.229785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.230461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.230604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.231177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.231456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.231771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.232169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.232651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.236567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.236567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.237794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.276863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 865/1739] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:32.281661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.317557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4d2b5-db06-4dba-97b2-0a6a13e530c0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.324499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.325502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.325932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.326370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.327233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.329780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.350473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 866/1740] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:32.357165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.364244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 875/1741] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:32 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:32.366199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.372441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c192-14c5-4c01-bead-39706d7bf54c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.543824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973f858-7fe5-40a1-848f-52ef7acefcf3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.546389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.547128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.547720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.547852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] PATH_INFO: `/v3/projects/aaf97670cebb4176ba40bc0feb8a738b/users/4cfb3add66ff445ca2c0ab60405c4508/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.558152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318614b3-459e-440a-9abd-daec0da5eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.559261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318614b3-459e-440a-9abd-daec0da5eab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.559261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318614b3-459e-440a-9abd-daec0da5eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.559717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318614b3-459e-440a-9abd-daec0da5eab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.560628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318614b3-459e-440a-9abd-daec0da5eab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.570811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318614b3-459e-440a-9abd-daec0da5eab7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.571010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf97670cebb4176ba40bc0feb8a738b, user_id=4cfb3add66ff445ca2c0ab60405c4508, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.571115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.589097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:32.669586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 867/1742] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:32 2022] PUT /identity/v3/projects/aaf97670cebb4176ba40bc0feb8a738b/users/4cfb3add66ff445ca2c0ab60405c4508/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:32.670624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.690439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 876/1743] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:32 2022] POST /identity/v3/users => generated 290 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:32.691793 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237453a-048c-4623-a9e1-903382548f11 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.692068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237453a-048c-4623-a9e1-903382548f11 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.692366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237453a-048c-4623-a9e1-903382548f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.749452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7237453a-048c-4623-a9e1-903382548f11 None None] MFA Rules not processed for user `4cfb3add66ff445ca2c0ab60405c4508`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:32.750690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7237453a-048c-4623-a9e1-903382548f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:32.813096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c371a-4215-42b9-8f1e-bde479cde646 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.825615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.827379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.828090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.828844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.834502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:32.845983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 877/1744] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:32.846962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237453a-048c-4623-a9e1-903382548f11 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.946666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237453a-048c-4623-a9e1-903382548f11 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:32.956747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:32.957828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:32.958275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:32.958892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:32.959982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.962868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.975536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 878/1745] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:32.976909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:32.988400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:32.991360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:32.991654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a1e82-6128-44a2-aaa9-798f623b28b8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.004801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 868/1746] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:32 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:33.005800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.020038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.021589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.023230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.023611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.026342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.030293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.062734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:33.080212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652b4df-2784-4f6d-8776-64d90591a475 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.095554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.100115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.100115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.100115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.101018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.104597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.121318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 869/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:33.126384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.140401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124f2a-fe39-4cb4-b524-a87ff9c7549f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.146746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0490480-48b8-42c5-8374-b051ca36c44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.147766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490480-48b8-42c5-8374-b051ca36c44b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.148523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490480-48b8-42c5-8374-b051ca36c44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.149066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0490480-48b8-42c5-8374-b051ca36c44b admin admin] PATH_INFO: `/v3/projects/890dd7b368a94889afab0a0fc9b96fc6/users/242db9d8614c4efb9b79eb704f1c8797/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.172592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0490480-48b8-42c5-8374-b051ca36c44b admin admin] RBAC: Authorizing `identity:create_grant(project_id=890dd7b368a94889afab0a0fc9b96fc6, user_id=242db9d8614c4efb9b79eb704f1c8797, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.176537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0490480-48b8-42c5-8374-b051ca36c44b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.221469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 879/1748] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:32 2022] POST /identity/v3/projects => generated 381 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:33.225112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.282959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 870/1749] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:33 2022] PUT /identity/v3/projects/890dd7b368a94889afab0a0fc9b96fc6/users/242db9d8614c4efb9b79eb704f1c8797/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:33.304218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.308788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.310328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.372002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] MFA Rules not processed for user `242db9d8614c4efb9b79eb704f1c8797`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:33.373332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:33.374980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a90db5-29d4-400c-a229-cf15a80cf7c2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.557174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.557540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.557827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.558931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.558931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.563725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.636796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 871/1750] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:33.637847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.655853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06234d5-7537-4506-b6b6-8871b59b04a9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.670787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.671420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.671696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.672105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.672667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.674737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.705755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 880/1751] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:33 2022] POST /identity/v3/users => generated 297 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:33.710029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.731203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:33.731364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b13fa9-6262-4548-95ca-9af373569321 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.744584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.745262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.745604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.745919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.746681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.748420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.773851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 881/1752] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:44:33.777752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.788873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09541d-d583-493b-ab60-20e9b1ab53b4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.794599 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d1f8f-2085-4b0d-93fb-c764dee16ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.795273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1f8f-2085-4b0d-93fb-c764dee16ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.795742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1f8f-2085-4b0d-93fb-c764dee16ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.800860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1f8f-2085-4b0d-93fb-c764dee16ad2 admin admin] PATH_INFO: `/v3/projects/b09830ddcc3a4f74bb281566a26b9861/users/b39654bb8e394f61ab7b5d96f974408c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.802169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 872/1753] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:33 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:33.806133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:33.818329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d1f8f-2085-4b0d-93fb-c764dee16ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09830ddcc3a4f74bb281566a26b9861, user_id=b39654bb8e394f61ab7b5d96f974408c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.821735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d1f8f-2085-4b0d-93fb-c764dee16ad2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.898518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2687a1-e353-4aad-8cdc-e4c1f6e6a983 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:33.904467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:33.905034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.905306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.905731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.906478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:33.908665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:33.953229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 882/1754] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:33 2022] PUT /identity/v3/projects/b09830ddcc3a4f74bb281566a26b9861/users/b39654bb8e394f61ab7b5d96f974408c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:33.954644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:33.954975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:33.955251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:33.977513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] MFA Rules not processed for user `1d9f7755863d45bfb04a1498bd62e367`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:33.979271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:34.014129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 873/1755] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:33 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:44:34.016311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.028594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18eddf4-3e2d-4bcd-8eed-4e97ce315512 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:34.052641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 883/1756] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:34.054060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.109249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:34.109249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.109249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.109249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.109570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:34.111397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:34.134900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 874/1757] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:34.138022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.157329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde9c3e-6b2b-49fa-8eb5-fab84301d03c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:34.239672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3edc68-b94b-4152-8dc6-35ed80670682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:34.240728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3edc68-b94b-4152-8dc6-35ed80670682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.242682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3edc68-b94b-4152-8dc6-35ed80670682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.242984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3edc68-b94b-4152-8dc6-35ed80670682 admin admin] PATH_INFO: `/v3/projects/b09830ddcc3a4f74bb281566a26b9861/users/b39654bb8e394f61ab7b5d96f974408c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.249529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3edc68-b94b-4152-8dc6-35ed80670682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b09830ddcc3a4f74bb281566a26b9861, user_id=b39654bb8e394f61ab7b5d96f974408c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:34.264838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3edc68-b94b-4152-8dc6-35ed80670682 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:34.344839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 875/1758] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:34 2022] PUT /identity/v3/projects/b09830ddcc3a4f74bb281566a26b9861/users/b39654bb8e394f61ab7b5d96f974408c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:34.346427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.346630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cb6ed-b24d-4222-8b1b-64e3e4fa0e8c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:34.347031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.347031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.438239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] MFA Rules not processed for user `b39654bb8e394f61ab7b5d96f974408c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:34.439753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:34.501386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:34.502486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.503663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.503811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.504510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:34.508462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:34.555458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 884/1759] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:34.560114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.592365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781abf7-4a18-4a36-bc88-cf0b0ccedde8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:34.598099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 876/1760] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:34.611002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:34.611403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.611705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.612017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.612438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] PATH_INFO: `/v3/projects/f9804a18da8442c98b9c90b895c3613a/users/e97dbcdb42234238b1ded99adbe99b59/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.642162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9804a18da8442c98b9c90b895c3613a, user_id=e97dbcdb42234238b1ded99adbe99b59, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:34.644912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:34.754293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 885/1761] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:34 2022] PUT /identity/v3/projects/f9804a18da8442c98b9c90b895c3613a/users/e97dbcdb42234238b1ded99adbe99b59/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:34.755408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.835658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede958b-5b27-4096-b71d-f3dab6960b17 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:34.909538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda0420-f440-4009-b369-f0b05179e0ee admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:34.923443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:34.923898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.924182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.924487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.925145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:34.926825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:34.950346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:34.950975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:34.951264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:34.951535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:34.966577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 886/1762] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:44:34.968129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:34.984941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce05a1-e5a3-4681-81e8-f213a7e9f76d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:35.010698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b32b69-1c1c-412b-807d-b4d7f16f66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:35.011468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32b69-1c1c-412b-807d-b4d7f16f66e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:35.012451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32b69-1c1c-412b-807d-b4d7f16f66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:35.012766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b32b69-1c1c-412b-807d-b4d7f16f66e0 admin admin] PATH_INFO: `/v3/projects/f9804a18da8442c98b9c90b895c3613a/users/e97dbcdb42234238b1ded99adbe99b59/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:35.017529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32b69-1c1c-412b-807d-b4d7f16f66e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9804a18da8442c98b9c90b895c3613a, user_id=e97dbcdb42234238b1ded99adbe99b59, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:35.026150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b32b69-1c1c-412b-807d-b4d7f16f66e0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:35.080258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 887/1763] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:34 2022] PUT /identity/v3/projects/f9804a18da8442c98b9c90b895c3613a/users/e97dbcdb42234238b1ded99adbe99b59/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:35.082656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:35.083055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:35.083464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:35.180444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] MFA Rules not processed for user `e97dbcdb42234238b1ded99adbe99b59`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:35.182545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:35.189451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:35.191177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:35.230083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 877/1764] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:35.377105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:35.385622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 888/1765] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:35.423123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:35.424633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:35.450892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1c506-00ff-4baa-98c8-e5aa0ad453c2 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:35.557697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:35.563869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa574675-9e8c-42d5-8478-05d196ab649e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:35.564576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa574675-9e8c-42d5-8478-05d196ab649e service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:35.564860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa574675-9e8c-42d5-8478-05d196ab649e service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:35.565130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa574675-9e8c-42d5-8478-05d196ab649e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:35.576815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa574675-9e8c-42d5-8478-05d196ab649e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:35.578445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa574675-9e8c-42d5-8478-05d196ab649e service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:35.590415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 878/1766] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:35.623640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:35.636324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ed244-acb6-4520-b590-fc94128cdeeb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:35.641103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36f6e5-662a-41e3-8d8e-59da62afab85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:35.641799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36f6e5-662a-41e3-8d8e-59da62afab85 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:35.642147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36f6e5-662a-41e3-8d8e-59da62afab85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:35.642511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36f6e5-662a-41e3-8d8e-59da62afab85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:35.763703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36f6e5-662a-41e3-8d8e-59da62afab85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:35.765504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36f6e5-662a-41e3-8d8e-59da62afab85 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:35.795395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 889/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:35.863874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211925-7b47-4b09-9578-e5d674fdc472 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:35.864537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211925-7b47-4b09-9578-e5d674fdc472 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:35.864817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211925-7b47-4b09-9578-e5d674fdc472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:35.946097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d1211925-7b47-4b09-9578-e5d674fdc472 None None] MFA Rules not processed for user `4ce12f8401f3479ba14e5e1c3133a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:35.949927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d1211925-7b47-4b09-9578-e5d674fdc472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:36.108391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 879/1768] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:36.333362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:36.416380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cdcd3-f3f3-4cd8-a4e8-9f83874e3528 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:36.416972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cdcd3-f3f3-4cd8-a4e8-9f83874e3528 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:36.416972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cdcd3-f3f3-4cd8-a4e8-9f83874e3528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:36.522810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c59cdcd3-f3f3-4cd8-a4e8-9f83874e3528 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:36.524961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c59cdcd3-f3f3-4cd8-a4e8-9f83874e3528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:36.613863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 890/1769] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:44:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:36.769053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77b17-12a0-4c09-ba5b-2315e7611f6d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:36.769675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77b17-12a0-4c09-ba5b-2315e7611f6d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:36.769675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77b17-12a0-4c09-ba5b-2315e7611f6d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:36.771116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 880/1770] 10.222.0.17 () {54 vars in 912 bytes} [Wed Aug 24 16:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:44:36.786002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:36.786283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:36.786767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:36.815555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] MFA Rules not processed for user `9bde9ceb3c7c46298626580b62e0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:36.817227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:36.883601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 891/1771] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:44:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:36.916375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d77b17-12a0-4c09-ba5b-2315e7611f6d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:36.918459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61d77b17-12a0-4c09-ba5b-2315e7611f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:36.948532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d77b17-12a0-4c09-ba5b-2315e7611f6d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:36.956922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:36.957548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:36.957827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:36.958109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:36.970146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:36.971877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:36.982221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 881/1772] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:37.014429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:37.015849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:37.184091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319ed60-77ce-46f0-8731-8790632c3ac3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:37.207078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43241435-3133-438d-9844-549fea7ec752 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:37.207078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43241435-3133-438d-9844-549fea7ec752 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:37.207078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43241435-3133-438d-9844-549fea7ec752 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:37.207078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43241435-3133-438d-9844-549fea7ec752 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:37.238922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43241435-3133-438d-9844-549fea7ec752 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:37.241199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43241435-3133-438d-9844-549fea7ec752 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:37.253573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 892/1773] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:44:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:44:37.327704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:37.349127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab962-fad8-478f-bf78-a9752af31902 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:37.358271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e97dbcdb42234238b1ded99adbe99b59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9804a18da8442c98b9c90b895c3613a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:37.358965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:37.359689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:37.360107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:37.361171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:37.362634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:37.367786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:37.392772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 882/1774] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:37.398165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43241435-3133-438d-9844-549fea7ec752 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:37.415695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43241435-3133-438d-9844-549fea7ec752 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:37.426343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e97dbcdb42234238b1ded99adbe99b59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9804a18da8442c98b9c90b895c3613a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:37.428382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:37.429169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:37.429751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:37.432018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:37.434848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:37.639416 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:37.664680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:37.674211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 893/1775] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:37 2022] POST /identity/v3/projects => generated 353 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:37.681553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee14ef4-cf78-4140-8518-cfffb890f901 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:37.845195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e97dbcdb42234238b1ded99adbe99b59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9804a18da8442c98b9c90b895c3613a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:37.846607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:37.847052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:37.847474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:37.851170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:37.851764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:37.856002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061ea7e-5c44-4dc1-b0d2-ee94ad020f8f tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:37.878080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 883/1776] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:44:38.093936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:38.116465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac4281-e0a1-4e18-b18a-7c5a3253bf4e tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:38.130941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e97dbcdb42234238b1ded99adbe99b59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9804a18da8442c98b9c90b895c3613a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:38.131604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:38.132314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:38.132598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] PATH_INFO: `/v3/projects/de536705e779439fae14c3cb39c68909` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:38.135875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorizing `identity:delete_project(project_id=de536705e779439fae14c3cb39c68909)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:38.139251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:38.427280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de536705e779439fae14c3cb39c68909', 'request_id': 'req-deb2a25e-d493-4406-9550-9570063b4cc6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:38.447471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de536705e779439fae14c3cb39c68909', 'request_id': 'req-deb2a25e-d493-4406-9550-9570063b4cc6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:38.589195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de536705e779439fae14c3cb39c68909', 'request_id': 'req-deb2a25e-d493-4406-9550-9570063b4cc6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:38.625090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] The token cache is being invalidate because project de536705e779439fae14c3cb39c68909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:38.625090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:38.625090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:38.625090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 894/1777] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:38 2022] DELETE /identity/v3/projects/de536705e779439fae14c3cb39c68909 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:39.292584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:39.292584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:39.292584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:39.338841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] MFA Rules not processed for user `9bde9ceb3c7c46298626580b62e0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:39.361535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:39.495393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 884/1778] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Aug 24 16:44:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:39.577515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:39.580158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:39.616637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a25e-d493-4406-9550-9570063b4cc6 tempest-VolumeQuotaClassesTest-439215967 tempest-VolumeQuotaClassesTest-439215967-project-admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:39.627048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e856dac-eae1-4a57-9217-e129c7cc939c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:39.627932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e856dac-eae1-4a57-9217-e129c7cc939c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:39.628162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e856dac-eae1-4a57-9217-e129c7cc939c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:39.628488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e856dac-eae1-4a57-9217-e129c7cc939c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:39.638415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e856dac-eae1-4a57-9217-e129c7cc939c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:39.639927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e856dac-eae1-4a57-9217-e129c7cc939c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:39.667810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 895/1779] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:40.337780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:40.975640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:40.978005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:41.156943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decceaff-fe0d-4042-aa90-3547041d3651 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:41.168045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:41.169027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:41.169601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:41.170085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] PATH_INFO: `/v3/users/242db9d8614c4efb9b79eb704f1c8797` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:41.171888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] RBAC: Authorizing `identity:delete_user(user_id=242db9d8614c4efb9b79eb704f1c8797)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:41.178125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:41.512313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242db9d8614c4efb9b79eb704f1c8797'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:41.576703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:41.638616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242db9d8614c4efb9b79eb704f1c8797'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:41.638616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242db9d8614c4efb9b79eb704f1c8797'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:41.643090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e77db-dfff-47ef-a491-c3b702dba2c3 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:41.643786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e77db-dfff-47ef-a491-c3b702dba2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:41.644248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e77db-dfff-47ef-a491-c3b702dba2c3 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:41.645247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242db9d8614c4efb9b79eb704f1c8797'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:41.646526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 896/1780] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:44:41.675521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:41.676015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:41.676487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:41.697888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0598b40b-bfd0-447e-8a3a-cc5d4633737e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242db9d8614c4efb9b79eb704f1c8797'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:41.730925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 885/1781] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:40 2022] DELETE /identity/v3/users/242db9d8614c4efb9b79eb704f1c8797 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:41.737112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d447d8-8a58-49df-9161-10479362004f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:41.744217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d447d8-8a58-49df-9161-10479362004f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:41.744217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d447d8-8a58-49df-9161-10479362004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:41.761478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:41.765916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:41.820378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:41.841983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-67d447d8-8a58-49df-9161-10479362004f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:41.849194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-67d447d8-8a58-49df-9161-10479362004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:41.912743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 897/1782] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:44:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:42.049366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 886/1783] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:42.066366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:42.067940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:42.263083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92710cf-a4c2-4a12-979d-75eab8c56a96 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:42.274355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:42.274991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:42.275264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:42.275542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:42.303170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:42.305869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:42.327315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 898/1784] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:42.461031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d447d8-8a58-49df-9161-10479362004f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:42.464805 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-67d447d8-8a58-49df-9161-10479362004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:42.480628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:42.482439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:42.503952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d447d8-8a58-49df-9161-10479362004f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:42.518021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0143b-92ef-4ecd-8cd5-b28d5fbe6ed2 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:42.521141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:42.522907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:42.525151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:42.526378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:42.528234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:42.529859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:42.530129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:42.530844 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] PATH_INFO: `/v3/users/644b5471a1424632933948e49a381ee2` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:42.532321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=644b5471a1424632933948e49a381ee2)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:42.537781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:42.546920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:42.557816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:42.586205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 887/1785] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:42.829360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644b5471a1424632933948e49a381ee2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:42.830714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644b5471a1424632933948e49a381ee2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:42.840021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:42.841685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:42.846075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644b5471a1424632933948e49a381ee2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:42.858154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfd201-b600-42b6-8a7b-7bbf0b056a7c service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:42.863801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:42.864518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:42.864818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:42.865076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] PATH_INFO: `/v3/users/a30d975e84924d09b05165d605f95d8d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:42.866280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30d975e84924d09b05165d605f95d8d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:42.869994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:43.021680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644b5471a1424632933948e49a381ee2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.022320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8fd8a5f-f85d-42b1-9aad-a8593da7a98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644b5471a1424632933948e49a381ee2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.053331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 899/1786] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:42 2022] DELETE /identity/v3/users/644b5471a1424632933948e49a381ee2 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:43.060504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.060504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.060504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.082718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:43.084168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:43.138721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30d975e84924d09b05165d605f95d8d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.172998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 900/1787] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:43.182458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:43.189370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:43.256409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30d975e84924d09b05165d605f95d8d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.257346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30d975e84924d09b05165d605f95d8d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.269768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30d975e84924d09b05165d605f95d8d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.304127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30d975e84924d09b05165d605f95d8d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.311826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cd13-837d-4238-9f85-32c5ca045c01 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:43.319718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 888/1788] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:42 2022] DELETE /identity/v3/users/a30d975e84924d09b05165d605f95d8d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:43.320622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:43.325610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:43.326442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:43.326838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.330240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.330532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.332574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:43.354244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:43.356380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f510b-579b-45d8-987f-496d1ef683d9 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:43.374017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 901/1789] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:43.380238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.380238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.380238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.398163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:43.399999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:43.402472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1c43d-a91b-4a6c-b671-317b64d37f74 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:43.409693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:43.414538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.414538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.414538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] PATH_INFO: `/v3/projects/890dd7b368a94889afab0a0fc9b96fc6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.414538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=890dd7b368a94889afab0a0fc9b96fc6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:43.416032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 902/1790] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:43.416747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:43.423435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:43.424983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:43.504228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b42423-9084-4060-b5e4-8b0c46adef86 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:43.508467 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:43.509362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.509638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.510027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.525285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:43.527416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:43.543240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 903/1791] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:44:43.559852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890dd7b368a94889afab0a0fc9b96fc6', 'request_id': 'req-671d9e51-a73e-4789-af4f-e4467f0daf75'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.634625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890dd7b368a94889afab0a0fc9b96fc6', 'request_id': 'req-671d9e51-a73e-4789-af4f-e4467f0daf75'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.643216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890dd7b368a94889afab0a0fc9b96fc6', 'request_id': 'req-671d9e51-a73e-4789-af4f-e4467f0daf75'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.657902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] The token cache is being invalidate because project 890dd7b368a94889afab0a0fc9b96fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:43.658497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.659294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-671d9e51-a73e-4789-af4f-e4467f0daf75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:43.661637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 889/1792] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:43 2022] DELETE /identity/v3/projects/890dd7b368a94889afab0a0fc9b96fc6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:43.666228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:43.683342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cf3ac-2b05-489c-9d01-24fed7294b1e service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:43.732758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:43.764902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:43.766252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.766252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.766427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] PATH_INFO: `/v3/users/e97dbcdb42234238b1ded99adbe99b59` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.768376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97dbcdb42234238b1ded99adbe99b59)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:43.774557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:43.797105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:43.956063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0124b-bb1e-47b4-8aa2-3d1bdd2ff306 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.956256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0124b-bb1e-47b4-8aa2-3d1bdd2ff306 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.956557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0124b-bb1e-47b4-8aa2-3d1bdd2ff306 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:43.959143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 890/1793] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:44:43.973672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:43.974094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:43.974470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:44.034532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97dbcdb42234238b1ded99adbe99b59'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.035185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97dbcdb42234238b1ded99adbe99b59'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.048558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:44.055832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:44.065561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97dbcdb42234238b1ded99adbe99b59'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.129648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97dbcdb42234238b1ded99adbe99b59'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.130302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97dbcdb42234238b1ded99adbe99b59'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.163102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 904/1794] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:43 2022] DELETE /identity/v3/users/e97dbcdb42234238b1ded99adbe99b59 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:44.183047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 891/1795] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:44:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:44.202535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:44.205924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:44.300230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:44.307414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:44.328807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c276e-fa34-4227-aa40-80d62b03bdf1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:44.355763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04253a-2ee6-4dd6-b9aa-ee1bd4d424c1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:44.374871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:44.375696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:44.376239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:44.376698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:44.390668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:44.392439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:44.408584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 905/1796] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:44.410833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:44.412939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:44.427765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:44.428721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:44.429170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:44.429704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] PATH_INFO: `/v3/projects/d5a9718e3f7d4d96b14b2fdb21b1e324` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:44.432030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a9718e3f7d4d96b14b2fdb21b1e324)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:44.434937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:44.517695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8b0b5-2cf8-49f7-ab7f-2c7686ce7350 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:44.522353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:44.522992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:44.523270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:44.523538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] PATH_INFO: `/v3/projects/356c77a6ac98435f8aca0b1b61def246` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:44.525358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=356c77a6ac98435f8aca0b1b61def246)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:44.527273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:44.618209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a9718e3f7d4d96b14b2fdb21b1e324', 'request_id': 'req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.652720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356c77a6ac98435f8aca0b1b61def246', 'request_id': 'req-e0cd678f-0db0-4cf8-915a-371400b1e719'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.658969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356c77a6ac98435f8aca0b1b61def246', 'request_id': 'req-e0cd678f-0db0-4cf8-915a-371400b1e719'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.695411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a9718e3f7d4d96b14b2fdb21b1e324', 'request_id': 'req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.703113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a9718e3f7d4d96b14b2fdb21b1e324', 'request_id': 'req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.714746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] The token cache is being invalidate because project d5a9718e3f7d4d96b14b2fdb21b1e324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:44.715034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.715693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.717266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 892/1797] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:44 2022] DELETE /identity/v3/projects/d5a9718e3f7d4d96b14b2fdb21b1e324 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:44.721015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:44.722304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:44.769543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356c77a6ac98435f8aca0b1b61def246', 'request_id': 'req-e0cd678f-0db0-4cf8-915a-371400b1e719'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.785337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e6d5e-2f55-470b-8e8c-f43802b51afb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:44.786488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] The token cache is being invalidate because project 356c77a6ac98435f8aca0b1b61def246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:44.786782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.787449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:44.789058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 906/1798] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:44 2022] DELETE /identity/v3/projects/356c77a6ac98435f8aca0b1b61def246 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:44.793676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:44.795153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:44.872278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd678f-0db0-4cf8-915a-371400b1e719 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:44.874452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:44.875537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:44.875823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:44.876097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] PATH_INFO: `/v3/users/780615da99fa4b299f0eff600fec6c6c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:44.877697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=780615da99fa4b299f0eff600fec6c6c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:44.878157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:44.878805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:44.879084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:44.879354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] PATH_INFO: `/v3/users/88e1847f005b4ff6bd2dc3e7846bc130` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:44.879655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:44.880593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e1847f005b4ff6bd2dc3e7846bc130)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:44.882932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:44.935744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:45.016082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:45.089438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e1847f005b4ff6bd2dc3e7846bc130'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.090295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e1847f005b4ff6bd2dc3e7846bc130'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.099275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e1847f005b4ff6bd2dc3e7846bc130'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.105648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780615da99fa4b299f0eff600fec6c6c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.165964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e1847f005b4ff6bd2dc3e7846bc130'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.166908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e1847f005b4ff6bd2dc3e7846bc130'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.207727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 907/1799] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:44 2022] DELETE /identity/v3/users/88e1847f005b4ff6bd2dc3e7846bc130 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:45.213808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:45.221270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:45.250459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780615da99fa4b299f0eff600fec6c6c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.251761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780615da99fa4b299f0eff600fec6c6c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.269655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780615da99fa4b299f0eff600fec6c6c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.304749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1f87b718-6870-4889-b630-6d067db3ee46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780615da99fa4b299f0eff600fec6c6c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.318502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 893/1800] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:44 2022] DELETE /identity/v3/users/780615da99fa4b299f0eff600fec6c6c => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:45.340321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f89113-7c91-4739-b10b-3ebecc5a365e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:45.458901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:45.459603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:45.461068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:45.462042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] PATH_INFO: `/v3/projects/f9804a18da8442c98b9c90b895c3613a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:45.463602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9804a18da8442c98b9c90b895c3613a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:45.490695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:45.670001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9804a18da8442c98b9c90b895c3613a', 'request_id': 'req-21fdeb0b-629d-4f20-b112-6a220be8b726'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.680931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9804a18da8442c98b9c90b895c3613a', 'request_id': 'req-21fdeb0b-629d-4f20-b112-6a220be8b726'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.803673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9804a18da8442c98b9c90b895c3613a', 'request_id': 'req-21fdeb0b-629d-4f20-b112-6a220be8b726'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.829111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] The token cache is being invalidate because project f9804a18da8442c98b9c90b895c3613a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:45.830583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.831497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-21fdeb0b-629d-4f20-b112-6a220be8b726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:45.834445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 908/1801] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:45 2022] DELETE /identity/v3/projects/f9804a18da8442c98b9c90b895c3613a => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:45.839593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:45.839875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:45.840152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:45.864265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:45.865539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:45.971410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 894/1802] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:45.983258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:45.989959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:45.991455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.041620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:46.041620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:46.049875 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 909/1803] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:46.054168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:46.077600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:46.081293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:46.106247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16954f32-d816-4965-9fce-aefe4c954ad5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:46.111929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa535290-3ff4-4971-96e3-8576474744a5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:46.121209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:46.121855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.122150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.122365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.123083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:46.123395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:46.126837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:46.130398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:46.131047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.131328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.131600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] PATH_INFO: `/v3/projects/b90498cf7cca4d938c8cdb4e3bf4b015` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.149318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90498cf7cca4d938c8cdb4e3bf4b015)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:46.154962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:46.169156 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 895/1804] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:46.170786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:46.187349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:46.215659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cbb1f-7679-46e3-8c2c-4c5e21419e4f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:46.221842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:46.222916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.223336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.223739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] PATH_INFO: `/v3/projects/50aec55da7c24c7d92e1b82be498b993` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.225271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50aec55da7c24c7d92e1b82be498b993)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:46.229002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:46.326238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90498cf7cca4d938c8cdb4e3bf4b015', 'request_id': 'req-ad24b61e-2981-43c6-b473-5f567eeee93a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.347453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90498cf7cca4d938c8cdb4e3bf4b015', 'request_id': 'req-ad24b61e-2981-43c6-b473-5f567eeee93a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.370799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:46.390815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50aec55da7c24c7d92e1b82be498b993', 'request_id': 'req-4b06bb59-fe32-4353-81c4-0d298192ce23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.477397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90498cf7cca4d938c8cdb4e3bf4b015', 'request_id': 'req-ad24b61e-2981-43c6-b473-5f567eeee93a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.501165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] The token cache is being invalidate because project b90498cf7cca4d938c8cdb4e3bf4b015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:46.501457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.502150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.502695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:46.505446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 910/1805] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:46 2022] DELETE /identity/v3/projects/b90498cf7cca4d938c8cdb4e3bf4b015 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:46.506229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:46.507850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:46.535820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50aec55da7c24c7d92e1b82be498b993', 'request_id': 'req-4b06bb59-fe32-4353-81c4-0d298192ce23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.546411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50aec55da7c24c7d92e1b82be498b993', 'request_id': 'req-4b06bb59-fe32-4353-81c4-0d298192ce23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.572997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] The token cache is being invalidate because project 50aec55da7c24c7d92e1b82be498b993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:46.573656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.574732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b06bb59-fe32-4353-81c4-0d298192ce23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:46.576768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 896/1806] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:46 2022] DELETE /identity/v3/projects/50aec55da7c24c7d92e1b82be498b993 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:46.578492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7459c3c-5f9c-4b14-825e-1ca2235870ab None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.578963 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7459c3c-5f9c-4b14-825e-1ca2235870ab None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.579453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7459c3c-5f9c-4b14-825e-1ca2235870ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.613392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f7459c3c-5f9c-4b14-825e-1ca2235870ab None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:46.615098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f7459c3c-5f9c-4b14-825e-1ca2235870ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:46.666877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24b61e-2981-43c6-b473-5f567eeee93a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:46.761241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe921df-f69f-4c6b-85ca-204634500b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:46.761993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe921df-f69f-4c6b-85ca-204634500b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.762748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe921df-f69f-4c6b-85ca-204634500b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.763138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe921df-f69f-4c6b-85ca-204634500b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.766412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe921df-f69f-4c6b-85ca-204634500b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:46.770066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe921df-f69f-4c6b-85ca-204634500b87 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:46.831587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 897/1807] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:46.832878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ae90-da10-4a4e-b145-200a42f9678b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.833159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ae90-da10-4a4e-b145-200a42f9678b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.833321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9ae90-da10-4a4e-b145-200a42f9678b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.888328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-93c9ae90-da10-4a4e-b145-200a42f9678b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:46.890012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93c9ae90-da10-4a4e-b145-200a42f9678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:46.906540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 898/1808] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:46.919636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.919636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.919636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.973057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 911/1809] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:46 2022] POST /identity/v3/projects => generated 385 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:46.981716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:46.985591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:46.985591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:46.986023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:46.987393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:47.063395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:47.067089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:47.138912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 899/1810] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:47.140171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.154091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:47.210118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dae5b7-65f4-46f8-87e0-48b7f60193fe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.221575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.222547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.223360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.224384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.225477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.230816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.240595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 912/1811] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:47.241959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.267787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9344034-e25a-4149-9a95-2d55a8e76f40 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.274164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.274542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.274788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.275050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.275710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:47.276014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.278007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.293646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 913/1812] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:47.294368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.312664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1668f3-3540-496c-919f-985a192691ca admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.320237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.321592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.321868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.322135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.322791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:47.323204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.325329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.345146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 914/1813] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:47.346296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.362696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774b3b9-03fc-43eb-b85d-5a3d418bf723 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.373538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.374559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.375978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.377165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.391945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.391945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.411194 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 900/1814] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:47 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:47.412743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.439410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e53e9e-a7b0-4796-a261-40d5b265468d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.455552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.456335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.456880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.457480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.458431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.460715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.485792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 915/1815] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:47 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:47.485792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.607253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a935fd-91ac-4145-8b64-8d4119291d43 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.611166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 901/1816] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:47 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:47.612143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.760418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.763726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.764297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.764749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.767580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.770630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.806618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 916/1817] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:47.808764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:47.834856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca4cb-0f95-4891-a8d2-dfd06fa050cd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.847671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.849898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.850460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.852608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.853423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.857988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:47.952583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f767cf0e-b189-4553-bd15-17179753354d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:47.962835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:47.966234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:47.966695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:47.967187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:47.970687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:47.973417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.050586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 917/1818] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:47 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:48.051723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.075937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2405229-fde0-4da7-991f-27e1b9289b3b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.096373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.096373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.096373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.096373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.096808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.096808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.115542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 902/1819] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:47 2022] POST /identity/v3/users => generated 294 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:48.116577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.144560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd786964-1f21-41e8-9c3d-62a818842ed0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.146079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 918/1820] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:48.153312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.164232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.165075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.165492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.165853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.166803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.168733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.172893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:48.177312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2fdc8-3736-40ed-99d8-baa77317eab3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.190003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.191853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.192232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.192657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.193414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.197124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 903/1821] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:48.197360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.200536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.244243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 919/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:48.245829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.247592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ceb69c-4b4f-41b5-abb9-b0bdce33415d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.262413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.263652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.264285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.264991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] PATH_INFO: `/v3/projects/e178d1bca043413bae28b1103452ccc8/users/777929672d6e43238550b19ffb719e90/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.270357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1b908-f3a3-4030-b0f9-e55001b1df8f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.290877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.292464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.292851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.293611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.294490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.297416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.310824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e178d1bca043413bae28b1103452ccc8, user_id=777929672d6e43238550b19ffb719e90, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.318002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.330926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 920/1823] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:48.330926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.357222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60cc3b-73c2-4b0a-958e-c9fdf0d60d72 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.372980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae498ae-4499-40ea-89d0-b28ef6be51b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.373832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae498ae-4499-40ea-89d0-b28ef6be51b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.374266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae498ae-4499-40ea-89d0-b28ef6be51b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.375044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae498ae-4499-40ea-89d0-b28ef6be51b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.375874 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae498ae-4499-40ea-89d0-b28ef6be51b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.379154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae498ae-4499-40ea-89d0-b28ef6be51b9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.412073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 904/1824] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:48 2022] PUT /identity/v3/projects/e178d1bca043413bae28b1103452ccc8/users/777929672d6e43238550b19ffb719e90/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:48.427100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.454154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 921/1825] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:48.458830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.459118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.459399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.473289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:48.533089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] MFA Rules not processed for user `777929672d6e43238550b19ffb719e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:48.534607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:48.607792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77beef-887d-4ec0-b125-4ee3625ecdee admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.616099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.617017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.618065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.619054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] PATH_INFO: `/v3/projects/3e3b536597ee4e968817511fb69734c2/users/caabfe0cd8094f8aa060f0b16a1c3576/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.653014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e3b536597ee4e968817511fb69734c2, user_id=caabfe0cd8094f8aa060f0b16a1c3576, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.655771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.721147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 922/1826] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:48.722560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.730750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 905/1827] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:48 2022] PUT /identity/v3/projects/3e3b536597ee4e968817511fb69734c2/users/caabfe0cd8094f8aa060f0b16a1c3576/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:48.731724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:48.766577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b989d-9113-46af-9339-c507969d0b9c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.910117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde5cbf-6ecb-41d8-b90d-156177453a6d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:48.916624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8c23a-5449-4989-924d-2b0fa3166317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.917839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8c23a-5449-4989-924d-2b0fa3166317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.918389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8c23a-5449-4989-924d-2b0fa3166317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.918966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8c23a-5449-4989-924d-2b0fa3166317 admin admin] PATH_INFO: `/v3/projects/fbb5c95bb53648818332d234c8330ca9/users/50a2ec363194481980b8f603f936464a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.928124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315369f3-45c4-4c9a-a421-94a6fb2d0cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:48.928870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315369f3-45c4-4c9a-a421-94a6fb2d0cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:48.929290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315369f3-45c4-4c9a-a421-94a6fb2d0cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:48.930190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315369f3-45c4-4c9a-a421-94a6fb2d0cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:48.931413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315369f3-45c4-4c9a-a421-94a6fb2d0cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.936668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315369f3-45c4-4c9a-a421-94a6fb2d0cf4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:48.939320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8c23a-5449-4989-924d-2b0fa3166317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb5c95bb53648818332d234c8330ca9, user_id=50a2ec363194481980b8f603f936464a, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:48.944160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8c23a-5449-4989-924d-2b0fa3166317 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:49.044898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:49.088540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 923/1828] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:48 2022] PUT /identity/v3/projects/fbb5c95bb53648818332d234c8330ca9/users/50a2ec363194481980b8f603f936464a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:49.090070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.090363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.093148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.111486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 906/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:48 2022] POST /identity/v3/projects => generated 383 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:49.113114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.113326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.113601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.182558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] MFA Rules not processed for user `caabfe0cd8094f8aa060f0b16a1c3576`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:49.184799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:49.200520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] MFA Rules not processed for user `50a2ec363194481980b8f603f936464a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:49.201978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:49.359437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 924/1830] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:49.361157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:49.392678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 907/1831] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:49.394043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:49.502744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:49.507518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9930b-b71b-4b46-9ad3-c428450b9b3f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:49.514240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:49.514871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.515152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.515423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.516061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:49.518974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:49.559222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406ab98-d51e-4d83-989d-8d82ee0291d3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:49.578406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:49.579067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.579772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.580206 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.581252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:49.583275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:49.691157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 925/1832] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:49 2022] POST /identity/v3/users => generated 292 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:49.693133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:49.741487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 908/1833] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:49 2022] POST /identity/v3/projects => generated 381 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:49.746501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9dcb-a455-4561-ade3-7761c687ae6e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:49.747622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:49.844396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:49.845332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.845861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.846443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.847323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:49.855992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:49.857225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbb143-8cd9-49e0-9524-8c56cdf80b72 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:49.865367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:49.866563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.866906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.867279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.870937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:49.872530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:49.891923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 909/1834] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:49.892889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:49.913814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb83d5-920c-49ef-b3b5-68d32ae9ff11 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:49.922635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:49.924624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:49.925301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:49.926079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:49.927473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:49.932372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.052712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 926/1835] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:49 2022] POST /identity/v3/projects => generated 385 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:50.053859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.067630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 910/1836] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:49 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:50.067630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.094561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645344da-427e-4372-9ea5-6ee0023e6176 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.280035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.280835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.282027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.283555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.287880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.299155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.307407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b67d3-ff05-450f-8922-45f88be6b4e3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.314063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.315024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.315482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.315940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.318378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.324304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.334199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 927/1837] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:50.339527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.352281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d272d-f16b-48ea-8a34-ab5238a5116a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.357845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.359275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.359572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.360146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.360693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.362470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.384461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 928/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:50.385618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.419747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d5007-04b0-4720-bad6-f8c5dcce21ee admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.420967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 911/1839] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:50 2022] POST /identity/v3/users => generated 299 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:50.423440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.435642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.440988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.440988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.440988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] PATH_INFO: `/v3/projects/d722b580c239432dafef6b3605e89e53/users/7f5d03bc79474fd2a519baaeae22a5d5/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.465464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e19f9-3cd7-4050-8d9e-f388b41ce7ba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.487245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.490061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.497083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.497548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] PATH_INFO: `/v3/projects/3ffabd7ed87143f5a94cfe15a9ec9276/users/c8c020f552274224bfe9df05e18d2f27/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.498550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d722b580c239432dafef6b3605e89e53, user_id=7f5d03bc79474fd2a519baaeae22a5d5, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.509870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.545210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffabd7ed87143f5a94cfe15a9ec9276, user_id=c8c020f552274224bfe9df05e18d2f27, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.550329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.680058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 929/1840] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:50 2022] PUT /identity/v3/projects/d722b580c239432dafef6b3605e89e53/users/7f5d03bc79474fd2a519baaeae22a5d5/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:50.680058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.692215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 912/1841] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:50 2022] PUT /identity/v3/projects/3ffabd7ed87143f5a94cfe15a9ec9276/users/c8c020f552274224bfe9df05e18d2f27/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:50.697052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:50.699938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:50.856920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100b889-390e-40a6-928a-e505c7387e47 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.885031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5728b-7106-443b-a4cc-226cf0a0777e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:50.915099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.915750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.918593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.918894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] PATH_INFO: `/v3/users/4cfb3add66ff445ca2c0ab60405c4508` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.920179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cfb3add66ff445ca2c0ab60405c4508)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.922306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.931906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348ab8b-de48-41d5-bf34-fc2af8f6f27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:50.932632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348ab8b-de48-41d5-bf34-fc2af8f6f27b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.932940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348ab8b-de48-41d5-bf34-fc2af8f6f27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.933399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348ab8b-de48-41d5-bf34-fc2af8f6f27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:50.934401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348ab8b-de48-41d5-bf34-fc2af8f6f27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:50.937172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348ab8b-de48-41d5-bf34-fc2af8f6f27b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:50.955594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 930/1842] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:50 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:50.957660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:50.957972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:50.958266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.000146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] MFA Rules not processed for user `7f5d03bc79474fd2a519baaeae22a5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:51.001183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.165122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 931/1843] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:51.177240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.204733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f6cf5-436e-471d-ae9b-a80d0088b5b2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.205298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cfb3add66ff445ca2c0ab60405c4508'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.210571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.211222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.220523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.220523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.220523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.220523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.237206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 932/1844] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:51.238820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.269750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbaf727-dc6e-419d-8448-0ed931b4258b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.292785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.292996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.293348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.293515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] PATH_INFO: `/v3/projects/ecbccb517b414d7ea6787c25c3ed06e7/users/45bd0f7cddeb419abb4cdb29e65c5693/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.301573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cfb3add66ff445ca2c0ab60405c4508'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.302305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cfb3add66ff445ca2c0ab60405c4508'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.318591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbccb517b414d7ea6787c25c3ed06e7, user_id=45bd0f7cddeb419abb4cdb29e65c5693, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.323291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.340460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cfb3add66ff445ca2c0ab60405c4508'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.377945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cfb3add66ff445ca2c0ab60405c4508'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:51.401272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 913/1845] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:50 2022] DELETE /identity/v3/users/4cfb3add66ff445ca2c0ab60405c4508 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:51.402616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.405651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.425055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 933/1846] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:51 2022] PUT /identity/v3/projects/ecbccb517b414d7ea6787c25c3ed06e7/users/45bd0f7cddeb419abb4cdb29e65c5693/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:51.427224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:51.429187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.434773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.489975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:51.582773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d2ba5-ea77-4c49-be11-932a5fc9cf30 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.623833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650808-73fa-4203-8a2e-a5e3bcc7109c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.637266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ea355-97d0-49ea-93ec-ba1dfc03785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.638008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ea355-97d0-49ea-93ec-ba1dfc03785c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.641953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ea355-97d0-49ea-93ec-ba1dfc03785c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.641953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ea355-97d0-49ea-93ec-ba1dfc03785c admin admin] PATH_INFO: `/v3/projects/3ffabd7ed87143f5a94cfe15a9ec9276/users/c8c020f552274224bfe9df05e18d2f27/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.651005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ea355-97d0-49ea-93ec-ba1dfc03785c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffabd7ed87143f5a94cfe15a9ec9276, user_id=c8c020f552274224bfe9df05e18d2f27, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.652973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ea355-97d0-49ea-93ec-ba1dfc03785c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.676535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 934/1847] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:51 2022] PUT /identity/v3/projects/3ffabd7ed87143f5a94cfe15a9ec9276/users/c8c020f552274224bfe9df05e18d2f27/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:51.677948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.677948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.678119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.720802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9ca37-8b73-43e7-bd04-666c104adf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.720942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:51.721396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9ca37-8b73-43e7-bd04-666c104adf4d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.721761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9ca37-8b73-43e7-bd04-666c104adf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.722077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9ca37-8b73-43e7-bd04-666c104adf4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.722179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.722716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9ca37-8b73-43e7-bd04-666c104adf4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.725137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9ca37-8b73-43e7-bd04-666c104adf4d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.846964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 935/1848] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:51.846964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.848396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.871275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417065-16d9-444a-a359-9de2fa00a288 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:51.900934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:51.909766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.909766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.909766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.909766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:51.909766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:51.910782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 914/1849] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:51 2022] POST /identity/v3/projects => generated 385 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:51.910782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:51.910919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:51.911074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:51.939444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 936/1850] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:51.940631 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:51.942537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:51.968955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] MFA Rules not processed for user `c8c020f552274224bfe9df05e18d2f27`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:51.970408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:52.025927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada4884-5fdd-40bf-a947-fe359dfd4130 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.137361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 915/1851] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:52.138327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.148454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.149499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.149854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.150207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.243127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0144e0c-26cf-489b-b9c3-2b032a714115 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.253323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.254094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.254374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.254645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.255313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.257729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.263360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.265201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.278203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 937/1852] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:52.279105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.290957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115119a-bb2f-48e9-b5ef-dcb8c774fac3 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.296116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.296865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.297149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.297418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] PATH_INFO: `/v3/projects/ecbccb517b414d7ea6787c25c3ed06e7/users/45bd0f7cddeb419abb4cdb29e65c5693/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.299402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbccb517b414d7ea6787c25c3ed06e7, user_id=45bd0f7cddeb419abb4cdb29e65c5693, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.302739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.316775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 938/1853] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:52 2022] PUT /identity/v3/projects/ecbccb517b414d7ea6787c25c3ed06e7/users/45bd0f7cddeb419abb4cdb29e65c5693/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:52.543859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.543859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:52.543859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82403cba-ef32-4e65-86ee-feb17db76611 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.545973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861f00-49c9-4885-83db-8407405eeb61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.545973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861f00-49c9-4885-83db-8407405eeb61 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.545973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861f00-49c9-4885-83db-8407405eeb61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.545973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861f00-49c9-4885-83db-8407405eeb61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.546945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861f00-49c9-4885-83db-8407405eeb61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.546945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861f00-49c9-4885-83db-8407405eeb61 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.546945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 939/1854] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:52.546945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.546945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.546945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.547331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 916/1855] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:52 2022] POST /identity/v3/users => generated 299 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:52.549699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.561801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229d89a-d142-4c8a-ac69-d885bc4ea3bd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.570825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] MFA Rules not processed for user `45bd0f7cddeb419abb4cdb29e65c5693`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:52.572673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:52.644642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.645356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.645780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.646053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.646856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.649607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.664357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 917/1856] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:52.666729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.673541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 940/1857] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:52.681170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c765-c1ff-4b7e-82c6-002ee4a7658d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.684173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.686576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:52.690905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.691778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.692053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.692319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] PATH_INFO: `/v3/projects/b19761659a064e23acfd69163d5925af/users/d41b1b2400fa48b99fb8681b55fd07ea/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.702883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997dedf-584e-4f03-8781-2ec48d5223e2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:52.706950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:52.707622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:52.707902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:52.708170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:52.711702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19761659a064e23acfd69163d5925af, user_id=d41b1b2400fa48b99fb8681b55fd07ea, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.715320 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.718661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:52.720194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:52.730827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 941/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:52.859366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.874984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 918/1859] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:52 2022] PUT /identity/v3/projects/b19761659a064e23acfd69163d5925af/users/d41b1b2400fa48b99fb8681b55fd07ea/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:52.879110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:52.886210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9d2b6-cb17-407c-9aa9-7ca1a7ece0ee service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.047148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e90619-94d9-4e74-a5e6-18be7cc75b72 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.050609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.050609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.050609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.050609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] PATH_INFO: `/v3/users/50a2ec363194481980b8f603f936464a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.052617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a2ec363194481980b8f603f936464a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.055627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:53.071721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.071721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.071721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.071721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.072007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.072007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:53.116788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 919/1860] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:53.116788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:53.128848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6a430-94e5-4fc7-a847-3d6ea1f2485d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.145281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf6e2-ff8a-4ff8-8946-8a9d77317731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.146329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf6e2-ff8a-4ff8-8946-8a9d77317731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.146329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf6e2-ff8a-4ff8-8946-8a9d77317731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.146539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf6e2-ff8a-4ff8-8946-8a9d77317731 admin admin] PATH_INFO: `/v3/projects/b19761659a064e23acfd69163d5925af/users/d41b1b2400fa48b99fb8681b55fd07ea/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.149605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf6e2-ff8a-4ff8-8946-8a9d77317731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19761659a064e23acfd69163d5925af, user_id=d41b1b2400fa48b99fb8681b55fd07ea, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.152671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf6e2-ff8a-4ff8-8946-8a9d77317731 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:53.170908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 920/1861] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:53 2022] PUT /identity/v3/projects/b19761659a064e23acfd69163d5925af/users/d41b1b2400fa48b99fb8681b55fd07ea/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:53.187377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.187747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.188862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.229408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] MFA Rules not processed for user `d41b1b2400fa48b99fb8681b55fd07ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:53.231998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.332403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 921/1862] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:53.342208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:53.343648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.387389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a2ec363194481980b8f603f936464a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.389838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a2ec363194481980b8f603f936464a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.403634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a2ec363194481980b8f603f936464a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.443825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823a07e-6396-4b45-909f-19e2fd1c4901 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.449039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.449672 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.449976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.450250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] PATH_INFO: `/v3/projects/aaf97670cebb4176ba40bc0feb8a738b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.451377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf97670cebb4176ba40bc0feb8a738b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.453339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:53.478332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a2ec363194481980b8f603f936464a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.479138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a2ec363194481980b8f603f936464a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.497039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 942/1863] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:52 2022] DELETE /identity/v3/users/50a2ec363194481980b8f603f936464a => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:53.497902 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:53.513462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.566798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf97670cebb4176ba40bc0feb8a738b', 'request_id': 'req-cdc0090f-bd1c-4311-b693-c8c3f4415f56'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.574266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982607b-2046-4bd7-8c4b-c8d295f32cac admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.624284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.624933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.625196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.625466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.627196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.661200 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf97670cebb4176ba40bc0feb8a738b', 'request_id': 'req-cdc0090f-bd1c-4311-b693-c8c3f4415f56'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.671607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf97670cebb4176ba40bc0feb8a738b', 'request_id': 'req-cdc0090f-bd1c-4311-b693-c8c3f4415f56'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.693057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] The token cache is being invalidate because project aaf97670cebb4176ba40bc0feb8a738b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:53.701392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.701543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.701649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cdc0090f-bd1c-4311-b693-c8c3f4415f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:53.701737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:53.701827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 922/1864] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:53 2022] DELETE /identity/v3/projects/aaf97670cebb4176ba40bc0feb8a738b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:53.701827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.701827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.701827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.707591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 943/1865] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:53.708715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:53.710118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.725713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:53.727921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.802488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5fc7f-c574-4891-a23d-5c94f3986616 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.811041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.811675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.811953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.812221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] PATH_INFO: `/v3/users/caabfe0cd8094f8aa060f0b16a1c3576` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.813572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caabfe0cd8094f8aa060f0b16a1c3576)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.820110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:53.825321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 923/1866] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:53.825321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:53.832555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:53.845850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7515d72-f402-4906-9837-c590f4a10a4d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:53.850571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:53.851852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:53.852123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:53.852426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] PATH_INFO: `/v3/users/b39654bb8e394f61ab7b5d96f974408c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:53.853646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b39654bb8e394f61ab7b5d96f974408c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:53.855619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.057055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caabfe0cd8094f8aa060f0b16a1c3576'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.057268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b39654bb8e394f61ab7b5d96f974408c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.057745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caabfe0cd8094f8aa060f0b16a1c3576'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.065294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caabfe0cd8094f8aa060f0b16a1c3576'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.140978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b39654bb8e394f61ab7b5d96f974408c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.141488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:54.141660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b39654bb8e394f61ab7b5d96f974408c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.149998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b39654bb8e394f61ab7b5d96f974408c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.161812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b39654bb8e394f61ab7b5d96f974408c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.167886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 924/1867] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:53 2022] DELETE /identity/v3/users/b39654bb8e394f61ab7b5d96f974408c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:54.168697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.170058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.233929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caabfe0cd8094f8aa060f0b16a1c3576'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.235533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-51e34c58-45d5-4e6d-8f1b-c63f44e56ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caabfe0cd8094f8aa060f0b16a1c3576'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:54.235642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0755e5-d605-4463-b672-86579b472b18 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.238876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:54.249072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.249705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.249982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.250255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.254812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.272154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 944/1868] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:53 2022] DELETE /identity/v3/users/caabfe0cd8094f8aa060f0b16a1c3576 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:54.274854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.275147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.275422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.308642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:54.310461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.358484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.360680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.377051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 925/1869] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:54.387601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 945/1870] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:54.397290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.398804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.409478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271035-0ed1-4ffb-860a-ef966abd3100 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.476639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.476639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.476639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.476639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.486805 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.488447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.499861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 926/1871] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:44:54.869132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.871302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.887545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d853c4-3697-47bb-9db8-d2288c8ce58b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.903633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.904649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.904649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.904848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] PATH_INFO: `/v3/projects/b09830ddcc3a4f74bb281566a26b9861` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.906486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b09830ddcc3a4f74bb281566a26b9861)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.908476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:54.921504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:54.923644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:54.942499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8838c-97af-49a3-85a3-9286c3524091 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:54.949561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:54.950235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:54.951314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:54.951645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] PATH_INFO: `/v3/projects/fbb5c95bb53648818332d234c8330ca9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:54.953003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb5c95bb53648818332d234c8330ca9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:54.956562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.065693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b09830ddcc3a4f74bb281566a26b9861', 'request_id': 'req-df527b05-6f8c-4153-802e-ca247c80b593'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.086112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b09830ddcc3a4f74bb281566a26b9861', 'request_id': 'req-df527b05-6f8c-4153-802e-ca247c80b593'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.097390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb5c95bb53648818332d234c8330ca9', 'request_id': 'req-de229349-02fb-420f-b8a7-7eb024039cac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.215273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b09830ddcc3a4f74bb281566a26b9861', 'request_id': 'req-df527b05-6f8c-4153-802e-ca247c80b593'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.226773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] The token cache is being invalidate because project b09830ddcc3a4f74bb281566a26b9861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:55.229427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.229427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-df527b05-6f8c-4153-802e-ca247c80b593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.229427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 946/1872] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:54 2022] DELETE /identity/v3/projects/b09830ddcc3a4f74bb281566a26b9861 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:55.234036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.234323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.234624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.255218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:55.256820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.312193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb5c95bb53648818332d234c8330ca9', 'request_id': 'req-de229349-02fb-420f-b8a7-7eb024039cac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.319766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb5c95bb53648818332d234c8330ca9', 'request_id': 'req-de229349-02fb-420f-b8a7-7eb024039cac'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.322121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 947/1873] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:55.324138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.325499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.331920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] The token cache is being invalidate because project fbb5c95bb53648818332d234c8330ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:55.332207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.332876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-de229349-02fb-420f-b8a7-7eb024039cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.334407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 927/1874] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:54 2022] DELETE /identity/v3/projects/fbb5c95bb53648818332d234c8330ca9 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:44:55.335831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.336108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.336472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.340460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568a92f-fcab-457c-b398-ed7f8791b2c3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.345823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.346474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.346796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.347074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] PATH_INFO: `/v3/projects/3e3b536597ee4e968817511fb69734c2` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.348241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e3b536597ee4e968817511fb69734c2)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.350514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.356794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:55.358094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.366319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 928/1875] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:55.367953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.369506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.381081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e6341-0db9-4da2-932a-128c3526cc9e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.387069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.391662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.391662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.391662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] PATH_INFO: `/v3/users/45bd0f7cddeb419abb4cdb29e65c5693` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.391662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45bd0f7cddeb419abb4cdb29e65c5693)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.391662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.462452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e3b536597ee4e968817511fb69734c2', 'request_id': 'req-0d850724-80ea-43ec-a38c-416d32f6ff8f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.469077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e3b536597ee4e968817511fb69734c2', 'request_id': 'req-0d850724-80ea-43ec-a38c-416d32f6ff8f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.603926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e3b536597ee4e968817511fb69734c2', 'request_id': 'req-0d850724-80ea-43ec-a38c-416d32f6ff8f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.617364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] The token cache is being invalidate because project 3e3b536597ee4e968817511fb69734c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:55.617959 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.618498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.619573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45bd0f7cddeb419abb4cdb29e65c5693'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.622953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 948/1876] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:55 2022] DELETE /identity/v3/projects/3e3b536597ee4e968817511fb69734c2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:55.622953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.622953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.688901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d850724-80ea-43ec-a38c-416d32f6ff8f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.694075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.694939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45bd0f7cddeb419abb4cdb29e65c5693'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.695076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.695224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.695494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.695641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45bd0f7cddeb419abb4cdb29e65c5693'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.696138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:55.696486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.698119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.703381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45bd0f7cddeb419abb4cdb29e65c5693'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.710480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 949/1877] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:55.711375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.718122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.723598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45bd0f7cddeb419abb4cdb29e65c5693'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:55.732723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 929/1878] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:55 2022] DELETE /identity/v3/users/45bd0f7cddeb419abb4cdb29e65c5693 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:55.740876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.740876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:55.743258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb5c2a-77cb-4793-8bd2-5d00ba928fff admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.824230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e24c-65d5-4479-90b2-296481d40529 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:55.830400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.831027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.831464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.835955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.835955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.835955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.851385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:55.852166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:55.852764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:55.853134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] PATH_INFO: `/v3/users/c8c020f552274224bfe9df05e18d2f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:55.854469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c020f552274224bfe9df05e18d2f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:55.857941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:55.939628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 930/1879] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:55 2022] POST /identity/v3/projects => generated 365 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:55.940593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:55.942406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:56.003622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5dc8-f56e-4341-8da0-d484f2bf2187 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.008779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.012863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.012863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.012863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.012863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:56.077084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.078583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.088948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 931/1880] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:56.089766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.094435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c020f552274224bfe9df05e18d2f27'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.096107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c020f552274224bfe9df05e18d2f27'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.107765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c020f552274224bfe9df05e18d2f27'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.191990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87411b74-e170-4f74-9d52-d25b0ca51a41 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.203960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.204912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.205327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.208419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.208419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.209836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.215628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c020f552274224bfe9df05e18d2f27'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.216656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c020f552274224bfe9df05e18d2f27'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.250767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 950/1881] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:44:55 2022] DELETE /identity/v3/users/c8c020f552274224bfe9df05e18d2f27 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:56.323037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 932/1882] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:56 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:56.324269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.326639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:56.439522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84375699-d8a6-41c8-9c58-728cc6813fb8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.445765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.446482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.446843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.447194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.448052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.449785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.469620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:56.482266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 951/1883] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:56.482381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.495588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f1cca-bdc2-470b-b14a-fd68c426c66b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.505037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.507257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.508561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.510612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.511886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.529558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.532481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 933/1884] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:56.536128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.555428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291a14d-301c-4936-8aea-1e748202b0b1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.565631 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88e9eb-375a-4a10-b403-a2563e1912ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.566508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88e9eb-375a-4a10-b403-a2563e1912ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.569724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88e9eb-375a-4a10-b403-a2563e1912ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.569724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88e9eb-375a-4a10-b403-a2563e1912ef admin admin] PATH_INFO: `/v3/projects/b298da5171634796a806e44685bb87f1/users/2065486a12c748abb66834da1c5169f5/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.588261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:56.592880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88e9eb-375a-4a10-b403-a2563e1912ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b298da5171634796a806e44685bb87f1, user_id=2065486a12c748abb66834da1c5169f5, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.604196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88e9eb-375a-4a10-b403-a2563e1912ef admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.671712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.673992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:56.694342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d0ee-75cb-41e7-b5b3-a6ba4b8f5f93 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.713465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:56.714621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.715106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.715584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] PATH_INFO: `/v3/projects/ecbccb517b414d7ea6787c25c3ed06e7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.717121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbccb517b414d7ea6787c25c3ed06e7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:56.721632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:56.739034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 952/1885] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:56 2022] PUT /identity/v3/projects/b298da5171634796a806e44685bb87f1/users/2065486a12c748abb66834da1c5169f5/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:56.752057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:56.752616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:56.752989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:56.789922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] MFA Rules not processed for user `2065486a12c748abb66834da1c5169f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:56.791465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:56.865625 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 953/1886] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:56.873543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:56.907059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbccb517b414d7ea6787c25c3ed06e7', 'request_id': 'req-cd95e03f-3e1b-41b8-b582-b29e4eef2095'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.949001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380f1f3-e521-4bbe-9cb4-d5cd9f1fd03a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:56.986808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbccb517b414d7ea6787c25c3ed06e7', 'request_id': 'req-cd95e03f-3e1b-41b8-b582-b29e4eef2095'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:56.996617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbccb517b414d7ea6787c25c3ed06e7', 'request_id': 'req-cd95e03f-3e1b-41b8-b582-b29e4eef2095'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:57.008762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] The token cache is being invalidate because project ecbccb517b414d7ea6787c25c3ed06e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:57.009060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:57.009813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cd95e03f-3e1b-41b8-b582-b29e4eef2095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:57.011380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 934/1887] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:56 2022] DELETE /identity/v3/projects/ecbccb517b414d7ea6787c25c3ed06e7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:57.016498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493bd4f-2b7a-487f-9db4-3ce737f0eadc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.016804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493bd4f-2b7a-487f-9db4-3ce737f0eadc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.017059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493bd4f-2b7a-487f-9db4-3ce737f0eadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.023196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.023837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.024109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.024519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.025162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.030316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.038184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4493bd4f-2b7a-487f-9db4-3ce737f0eadc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:57.040838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4493bd4f-2b7a-487f-9db4-3ce737f0eadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:57.049424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 935/1888] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:57.058065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.058525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.058647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.079716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:57.081060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:57.090505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 936/1889] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:57.096410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.104928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75feb8b-57d3-4c09-8cff-9fe3e97fbcdd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.110321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.111321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.111682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.112016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.113443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:57.113918 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.120504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.143154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 937/1890] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:44:57.144567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.149361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 954/1891] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:56 2022] POST /identity/v3/projects => generated 365 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:57.151139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.153988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:57.156619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b607c9-c375-4521-ba7f-67bdb488e0e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.245873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232040cb-5d5f-4d37-8707-78c00de0fa9f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.259255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.259255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.259255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.259255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.267921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.268643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.268979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.269409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.270238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.270647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.272544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.273522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.397967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 938/1892] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:57 2022] POST /identity/v3/projects => generated 387 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:44:57.402624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 955/1893] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:57 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:44:57.404929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.405351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.420947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09debc-0e10-494d-969b-5101c1c71d5a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.512268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b509351-c30a-43ad-b751-ac4ea284fd21 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.517734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.518352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.518632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.518908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.519713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.521524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.522426 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.523693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.524044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.524415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.525157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.527152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.543559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 939/1894] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:57.545126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.557971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b0b61-70fd-4087-984c-5a7930134543 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.562865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.563662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.564053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.564458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] PATH_INFO: `/v3/projects/82e78e0ac65149109fd8420ca4914df1/users/5f7bdd833162414c9860fe1c863495ab/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.601977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e78e0ac65149109fd8420ca4914df1, user_id=5f7bdd833162414c9860fe1c863495ab, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.604871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.662690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 956/1895] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:57 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:57.665201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.672025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:57.680839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c669f-8e3d-4e63-8978-f9fedd5655b4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.686830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.687232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.687522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.687786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] PATH_INFO: `/v3/projects/3ffabd7ed87143f5a94cfe15a9ec9276` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.689324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffabd7ed87143f5a94cfe15a9ec9276)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.691721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.762758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 940/1896] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:57 2022] PUT /identity/v3/projects/82e78e0ac65149109fd8420ca4914df1/users/5f7bdd833162414c9860fe1c863495ab/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:57.762758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.901075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551a6a9-1e08-4071-9dc4-95bb24774c6d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.906582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.907385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.907646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.907924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.908638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.910268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.922150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 941/1897] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:57.923090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.933812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c726ca-5f2c-4c2b-a0ee-1d4abc262f31 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.938559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.939970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.940532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.940983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.942170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.945137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.974240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 942/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:57.974240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:57.974240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155f730-1a6f-4d67-a810-4481226d9f68 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:57.974240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:57.974962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:57.974962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:57.974962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:57.974962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:57.974962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:57.994673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 943/1899] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:44:57.994673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:58.002127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6a4a-8b60-44e8-9b2f-c54f1388e9be admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:58.007858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:58.008556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:58.008829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:58.009099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] PATH_INFO: `/v3/projects/82e78e0ac65149109fd8420ca4914df1/users/5f7bdd833162414c9860fe1c863495ab/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:58.011095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e78e0ac65149109fd8420ca4914df1, user_id=5f7bdd833162414c9860fe1c863495ab, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:58.013916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:58.025386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 944/1900] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:57 2022] PUT /identity/v3/projects/82e78e0ac65149109fd8420ca4914df1/users/5f7bdd833162414c9860fe1c863495ab/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:58.026160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:58.090355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a260e9-8987-4a06-a47e-9f33a6e25a7d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:58.095199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377ddac-1db8-4ddc-a5d7-b1d25ae91e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:58.095898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ddac-1db8-4ddc-a5d7-b1d25ae91e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:58.096180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ddac-1db8-4ddc-a5d7-b1d25ae91e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:58.096496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ddac-1db8-4ddc-a5d7-b1d25ae91e3f admin admin] PATH_INFO: `/v3/projects/7438ac2048694520bf58a14feac7b74a/users/91ef4febfef74e6399c58147798fc164/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:58.112183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377ddac-1db8-4ddc-a5d7-b1d25ae91e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7438ac2048694520bf58a14feac7b74a, user_id=91ef4febfef74e6399c58147798fc164, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:58.115094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377ddac-1db8-4ddc-a5d7-b1d25ae91e3f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:58.758285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 945/1901] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:44:58 2022] PUT /identity/v3/projects/7438ac2048694520bf58a14feac7b74a/users/91ef4febfef74e6399c58147798fc164/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:58.865075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d47eb4-76a0-475e-88ee-bd3b063bc1e9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:58.865075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d47eb4-76a0-475e-88ee-bd3b063bc1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:58.865075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d47eb4-76a0-475e-88ee-bd3b063bc1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:58.865971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffabd7ed87143f5a94cfe15a9ec9276', 'request_id': 'req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:58.865971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffabd7ed87143f5a94cfe15a9ec9276', 'request_id': 'req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:58.867490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-80d47eb4-76a0-475e-88ee-bd3b063bc1e9 None None] MFA Rules not processed for user `5f7bdd833162414c9860fe1c863495ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:58.867490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-80d47eb4-76a0-475e-88ee-bd3b063bc1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:58.901254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 946/1902] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:58.902503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:58.903128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:58.903411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:58.963001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] MFA Rules not processed for user `91ef4febfef74e6399c58147798fc164`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:58.964363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.102931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 947/1903] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:59.102931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.102931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.166309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d7f47-4d6c-4679-901d-c384b01a8b64 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:59.171364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:59.172161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.172643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.172993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.189808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:59.191278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:59.203188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 948/1904] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:59.203874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.205876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.219835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a145e1-663f-48a7-8564-416797b8378c service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:59.224476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:59.225236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.225637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.225835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.227767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.329125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:59.330637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:59.341726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 949/1905] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:44:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:44:59.342536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.413603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184611-80e1-4067-a17f-70e238b89c62 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:59.418393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:59.419042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.419316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.419582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.420272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:59.422315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:59.632705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffabd7ed87143f5a94cfe15a9ec9276', 'request_id': 'req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:59.643692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] The token cache is being invalidate because project 3ffabd7ed87143f5a94cfe15a9ec9276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:44:59.643982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:59.644675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b9bd9e73-bbe5-4759-8a90-ab8b0740be35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:44:59.646260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 957/1906] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:44:57 2022] DELETE /identity/v3/projects/3ffabd7ed87143f5a94cfe15a9ec9276 => generated 0 bytes in 1983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:44:59.651163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d2d77-3d30-49ff-9b12-5e822b8e98e5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.651475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d2d77-3d30-49ff-9b12-5e822b8e98e5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.651778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d2d77-3d30-49ff-9b12-5e822b8e98e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.671800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-560d2d77-3d30-49ff-9b12-5e822b8e98e5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:59.673069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-560d2d77-3d30-49ff-9b12-5e822b8e98e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.690306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 958/1907] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:44:59.691767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.692585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.692585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.716335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:44:59.716335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.728871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 959/1908] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:44:59.729926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.740271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a97c4-3c5c-4928-bf3b-f7ccf50f4e65 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:59.750378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:59.753519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.756548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.756548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.756548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:44:59.756548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:59.757162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:59.760908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 950/1909] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:59 2022] POST /identity/v3/projects => generated 387 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:59.764973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.766702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:44:59.773024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 960/1910] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:44:59.777853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.795313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33c27b-e7d5-440c-adbb-9e3520fede6c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:59.857017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9c7a-5d1e-4fa8-b0db-b924c33d1bee admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:44:59.876661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:59.878368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.878460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:44:59.878852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.878999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.879131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:44:59.879430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:44:59.879700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:59.879821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:44:59.880611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:44:59.882772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:59.883378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:44:59.939889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:44:59.975757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 951/1911] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:59 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:44:59.979644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.980576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 961/1912] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:44:59 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:44:59.984562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:44:59.993682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be995ae0-8f55-43bf-98ff-a81ef66a7069 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.065491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.066198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.066470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.066740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.067388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.069063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.074129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2333204-8304-4fd8-8129-b051767d8358 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.078583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.079197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.079528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.079802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.080685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.082963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.088972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 952/1913] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:00.091321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.103270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944a47d-7343-4076-ad1c-4ed9327bf648 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.108515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.109228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.109510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.109828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] PATH_INFO: `/v3/projects/e752c55972944ba99a33122e9cbf0826/users/f2df4693ba484d87a61c70bfb54b4ae1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.127788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e752c55972944ba99a33122e9cbf0826, user_id=f2df4693ba484d87a61c70bfb54b4ae1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.131504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.256482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 953/1914] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:00 2022] PUT /identity/v3/projects/e752c55972944ba99a33122e9cbf0826/users/f2df4693ba484d87a61c70bfb54b4ae1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:00.260863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.267099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 962/1915] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:44:59 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:00.268543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.283817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91389-3a65-4adb-8c80-b5a1fc91c3d7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.319284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:00.339019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e112173-0f99-45d3-96a4-3efcee5ae6e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.343876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.344531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.344938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.345213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.345899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.347514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.360988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.361192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 954/1916] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:00.361784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.366143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.366143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.366143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.367287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.369891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.386977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 963/1917] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:00.389666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e6419-77b5-4db2-817a-dbbefbd5f32f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.392655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.405612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17322b0c-6481-4782-a27b-52703041673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.406296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17322b0c-6481-4782-a27b-52703041673a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.406670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17322b0c-6481-4782-a27b-52703041673a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.406839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17322b0c-6481-4782-a27b-52703041673a admin admin] PATH_INFO: `/v3/projects/e752c55972944ba99a33122e9cbf0826/users/f2df4693ba484d87a61c70bfb54b4ae1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.408972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17322b0c-6481-4782-a27b-52703041673a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e752c55972944ba99a33122e9cbf0826, user_id=f2df4693ba484d87a61c70bfb54b4ae1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.412028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17322b0c-6481-4782-a27b-52703041673a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.418742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989287ab-c942-431a-92cf-4bb7c0f9e38c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.424600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 955/1918] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:00 2022] PUT /identity/v3/projects/e752c55972944ba99a33122e9cbf0826/users/f2df4693ba484d87a61c70bfb54b4ae1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:00.435216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.435519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.435771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.470529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] MFA Rules not processed for user `f2df4693ba484d87a61c70bfb54b4ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:00.471789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:00.496230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.496969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.497351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.497731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.498889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.500629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.519303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 964/1919] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:00.520295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.566926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf3d5-b322-4af9-b5b2-ffd3b86ca4c6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.588694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.588694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.588694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.588694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] PATH_INFO: `/v3/projects/141c7b588dae4d0287e785dd706766af/users/9eae4bd4274342f59aa67cd6d4861cb1/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.589270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 956/1920] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:00.589698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.591065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:00.614985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141c7b588dae4d0287e785dd706766af, user_id=9eae4bd4274342f59aa67cd6d4861cb1, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.618913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.654486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8cf64-dc0d-4352-a60b-0e939074ee9f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.658501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee271dd3-d754-4c64-8614-e928e994fa76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.659151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee271dd3-d754-4c64-8614-e928e994fa76 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.659507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee271dd3-d754-4c64-8614-e928e994fa76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.659837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee271dd3-d754-4c64-8614-e928e994fa76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.675813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee271dd3-d754-4c64-8614-e928e994fa76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.677379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee271dd3-d754-4c64-8614-e928e994fa76 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.689530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 957/1921] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:00.787964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 965/1922] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:00 2022] PUT /identity/v3/projects/141c7b588dae4d0287e785dd706766af/users/9eae4bd4274342f59aa67cd6d4861cb1/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:00.795511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.795511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.795756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.835882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] MFA Rules not processed for user `9eae4bd4274342f59aa67cd6d4861cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:00.837157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:00.859079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.932211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 958/1923] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:00.935270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:00.937490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:00.951833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9ca5c-d08c-43e6-9a37-8ff6de0dca63 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.955545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de522-7644-4eb2-8e4e-d48c80fd4457 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:00.960029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.961442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.961877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.962498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] PATH_INFO: `/v3/users/91ef4febfef74e6399c58147798fc164` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.962615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:00.963196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:00.963480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:00.963749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] PATH_INFO: `/v3/users/2065486a12c748abb66834da1c5169f5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:00.964556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ef4febfef74e6399c58147798fc164)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.968051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2065486a12c748abb66834da1c5169f5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:00.968051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:00.968401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:01.217765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ef4febfef74e6399c58147798fc164'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.218485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ef4febfef74e6399c58147798fc164'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.227007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ef4febfef74e6399c58147798fc164'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.258810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2065486a12c748abb66834da1c5169f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.362912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ef4febfef74e6399c58147798fc164'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.364004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ef4febfef74e6399c58147798fc164'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.390492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 966/1924] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:00 2022] DELETE /identity/v3/users/91ef4febfef74e6399c58147798fc164 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:01.390989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:01.393150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.439783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2065486a12c748abb66834da1c5169f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.440936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2065486a12c748abb66834da1c5169f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.451084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2065486a12c748abb66834da1c5169f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.468008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d13e5fad-5021-4085-aeeb-36195c5390c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2065486a12c748abb66834da1c5169f5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:01.482427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 959/1925] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:00 2022] DELETE /identity/v3/users/2065486a12c748abb66834da1c5169f5 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:01.492859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823977a5-d9b8-47cc-85fa-9b359d68dfcd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:01.493177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823977a5-d9b8-47cc-85fa-9b359d68dfcd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:01.493446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823977a5-d9b8-47cc-85fa-9b359d68dfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:01.505994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4ab91-cbdc-42e0-a968-688938523367 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:01.517511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-823977a5-d9b8-47cc-85fa-9b359d68dfcd None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:01.518749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-823977a5-d9b8-47cc-85fa-9b359d68dfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.586610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:01.586610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:01.586610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:01.586610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:01.587795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.622646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 960/1926] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:01.622646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:01.622646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:01.622646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:01.677587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:01.679236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.691847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 961/1927] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:01.693213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:01.695659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.717877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:01.723030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:01.793460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 967/1928] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:01.801650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:01.806943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.864914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e051367-34f5-4099-ae9c-7ebb73a4606f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:01.871215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:01.875378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:01.875378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:01.875378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:01.883907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:01.885750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:01.894796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87b280-f08e-4593-8629-bf0b19e8e1bc service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:01.898853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 962/1929] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:01.900699 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375dc0e-59f2-4a4e-a084-601923e5c9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:01.901331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375dc0e-59f2-4a4e-a084-601923e5c9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:01.901654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375dc0e-59f2-4a4e-a084-601923e5c9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:01.901926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375dc0e-59f2-4a4e-a084-601923e5c9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:01.913832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375dc0e-59f2-4a4e-a084-601923e5c9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:01.914208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:01.915463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375dc0e-59f2-4a4e-a084-601923e5c9d0 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:01.917076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:01.929840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 968/1930] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:01.947519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcc5ee-46c9-4e9b-a12c-7592d3f7a3e1 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:01.954068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:01.954822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:01.955278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:01.955609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] PATH_INFO: `/v3/users/9eae4bd4274342f59aa67cd6d4861cb1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:01.958098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eae4bd4274342f59aa67cd6d4861cb1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:01.960868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:02.159624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83841e-a3a8-4e50-9b8d-bd9aa6f5a9f9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.160182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83841e-a3a8-4e50-9b8d-bd9aa6f5a9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.160182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83841e-a3a8-4e50-9b8d-bd9aa6f5a9f9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.161831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 969/1931] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:45:02.173018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dda38-1c8a-4c19-950b-a40aa7009219 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.173903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dda38-1c8a-4c19-950b-a40aa7009219 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.173903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61dda38-1c8a-4c19-950b-a40aa7009219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.194395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f61dda38-1c8a-4c19-950b-a40aa7009219 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:02.195748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f61dda38-1c8a-4c19-950b-a40aa7009219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.232839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eae4bd4274342f59aa67cd6d4861cb1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.309096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 970/1932] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:45:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:02.346711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eae4bd4274342f59aa67cd6d4861cb1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.348497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eae4bd4274342f59aa67cd6d4861cb1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.361549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eae4bd4274342f59aa67cd6d4861cb1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.391267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eae4bd4274342f59aa67cd6d4861cb1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.405159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 963/1933] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:01 2022] DELETE /identity/v3/users/9eae4bd4274342f59aa67cd6d4861cb1 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:02.406640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.406993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.407277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.432672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:02.433911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.481630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:02.488378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.523196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 971/1934] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:02.524244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:02.525767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.538993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4099d9-fd3b-45cf-bd36-c56e024fef9c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:02.544159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:02.545154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.545574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.546005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] PATH_INFO: `/v3/projects/7438ac2048694520bf58a14feac7b74a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.547200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7438ac2048694520bf58a14feac7b74a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:02.549270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:02.583743 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b51702-c285-4a40-a8d0-66ffe262c878 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:02.590677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:02.591249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.591535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.591815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.593818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.606993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:02.608619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:02.620452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 964/1935] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:02.621309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:02.622610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.681068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7438ac2048694520bf58a14feac7b74a', 'request_id': 'req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.690664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7438ac2048694520bf58a14feac7b74a', 'request_id': 'req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.693332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f19edd-a867-4cb6-81d1-812447483918 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:02.748439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:02.749110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.749384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.749651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.774345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7438ac2048694520bf58a14feac7b74a', 'request_id': 'req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.789044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] The token cache is being invalidate because project 7438ac2048694520bf58a14feac7b74a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:02.789305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.790143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:02.791810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 972/1936] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:02 2022] DELETE /identity/v3/projects/7438ac2048694520bf58a14feac7b74a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:02.792554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:02.795535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:02.823606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:02.825512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:02.865140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 965/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:02.865693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55675f5c-4dd5-44d1-92c3-3117d3fcbbdb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:02.869079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:02.876043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:02.876806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.877156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.877497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] PATH_INFO: `/v3/projects/b298da5171634796a806e44685bb87f1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.879693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b298da5171634796a806e44685bb87f1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:02.884016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:02.894363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200b5df-39b4-49e5-ae60-bbc872524f0c service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:02.907087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:02.907538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:02.907817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:02.908087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] PATH_INFO: `/v3/users/f2df4693ba484d87a61c70bfb54b4ae1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:02.916716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2df4693ba484d87a61c70bfb54b4ae1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:02.918811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:02.985580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:03.053543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b298da5171634796a806e44685bb87f1', 'request_id': 'req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.073457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b298da5171634796a806e44685bb87f1', 'request_id': 'req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.163824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b298da5171634796a806e44685bb87f1', 'request_id': 'req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.181532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2df4693ba484d87a61c70bfb54b4ae1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.192322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] The token cache is being invalidate because project b298da5171634796a806e44685bb87f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:03.192636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.197512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.199387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 973/1938] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:02 2022] DELETE /identity/v3/projects/b298da5171634796a806e44685bb87f1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:03.203374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:03.204850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:03.278319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2df4693ba484d87a61c70bfb54b4ae1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.279101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2df4693ba484d87a61c70bfb54b4ae1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.289630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2df4693ba484d87a61c70bfb54b4ae1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.311536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9a30-b8c7-4589-ad9d-0ffedfcdc0ad admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:03.339933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:03.340887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:03.341623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2df4693ba484d87a61c70bfb54b4ae1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.342339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:03.343570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] PATH_INFO: `/v3/users/5f7bdd833162414c9860fe1c863495ab` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:03.350577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f7bdd833162414c9860fe1c863495ab)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:03.358203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:03.359670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 966/1939] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:02 2022] DELETE /identity/v3/users/f2df4693ba484d87a61c70bfb54b4ae1 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:03.364180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:03.595393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f7bdd833162414c9860fe1c863495ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.596226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f7bdd833162414c9860fe1c863495ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.607136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f7bdd833162414c9860fe1c863495ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.682394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f7bdd833162414c9860fe1c863495ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.683556 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f7bdd833162414c9860fe1c863495ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:03.707814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 974/1940] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:03 2022] DELETE /identity/v3/users/5f7bdd833162414c9860fe1c863495ab => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:03.783083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:03.784718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:03.861167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca1b86-9ef7-45da-b894-2d016eb448c9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:03.925284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:03.925924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:03.926204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:03.926477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] PATH_INFO: `/v3/projects/141c7b588dae4d0287e785dd706766af` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:03.927603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] RBAC: Authorizing `identity:delete_project(project_id=141c7b588dae4d0287e785dd706766af)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:03.930177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.006478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.008643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.025845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cc5c8-9555-4aa4-957f-e1108592ebe3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.036466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.037243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.037799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.039369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] PATH_INFO: `/v3/projects/e752c55972944ba99a33122e9cbf0826` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.041301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e752c55972944ba99a33122e9cbf0826)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.048810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.058706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141c7b588dae4d0287e785dd706766af', 'request_id': 'req-7cfab6ee-b5fd-4520-82e3-237d1128080b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.164531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141c7b588dae4d0287e785dd706766af', 'request_id': 'req-7cfab6ee-b5fd-4520-82e3-237d1128080b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.173760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141c7b588dae4d0287e785dd706766af', 'request_id': 'req-7cfab6ee-b5fd-4520-82e3-237d1128080b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.187807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] The token cache is being invalidate because project 141c7b588dae4d0287e785dd706766af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:04.188304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.189365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.190063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e752c55972944ba99a33122e9cbf0826', 'request_id': 'req-2bd854c9-fe20-4446-ac89-d619a8cffd47'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.200273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e752c55972944ba99a33122e9cbf0826', 'request_id': 'req-2bd854c9-fe20-4446-ac89-d619a8cffd47'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.204564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 967/1941] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:03 2022] DELETE /identity/v3/projects/141c7b588dae4d0287e785dd706766af => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:04.205946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.210559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.266144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e752c55972944ba99a33122e9cbf0826', 'request_id': 'req-2bd854c9-fe20-4446-ac89-d619a8cffd47'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.284370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] The token cache is being invalidate because project e752c55972944ba99a33122e9cbf0826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:04.285095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.285798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2bd854c9-fe20-4446-ac89-d619a8cffd47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.287786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 975/1942] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:04 2022] DELETE /identity/v3/projects/e752c55972944ba99a33122e9cbf0826 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:04.289636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfa28e-ae4c-4e20-b1ee-76d51793deff None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.289821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfa28e-ae4c-4e20-b1ee-76d51793deff None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.290101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfa28e-ae4c-4e20-b1ee-76d51793deff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.294097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfab6ee-b5fd-4520-82e3-237d1128080b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.311529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f9dfa28e-ae4c-4e20-b1ee-76d51793deff None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:04.312802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f9dfa28e-ae4c-4e20-b1ee-76d51793deff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.368976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.369629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.369906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.370190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] PATH_INFO: `/v3/projects/82e78e0ac65149109fd8420ca4914df1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.371441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e78e0ac65149109fd8420ca4914df1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.373443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.380672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 976/1943] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:04.382542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2f91b-b84e-43b6-87d3-1919a4aaf93d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.382969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2f91b-b84e-43b6-87d3-1919a4aaf93d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.383242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2f91b-b84e-43b6-87d3-1919a4aaf93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.404117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-67b2f91b-b84e-43b6-87d3-1919a4aaf93d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:04.405545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-67b2f91b-b84e-43b6-87d3-1919a4aaf93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.415421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 977/1944] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:04.416805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97802fd-ef00-4fea-a756-410043520da9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.417207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97802fd-ef00-4fea-a756-410043520da9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.417522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97802fd-ef00-4fea-a756-410043520da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.441943 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f97802fd-ef00-4fea-a756-410043520da9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:04.443792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f97802fd-ef00-4fea-a756-410043520da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.454147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 978/1945] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:04.455558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.455848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.456121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.476180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:04.477409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.485823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 979/1946] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:04.489217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.497591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f75be-cd6b-44ee-8e82-386e0581b2aa None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.499760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e78e0ac65149109fd8420ca4914df1', 'request_id': 'req-313b15b8-3cf3-4026-94f7-fb893d986819'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.552887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.554106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.554397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.554683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.555328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:04.555630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.557370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.562860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e78e0ac65149109fd8420ca4914df1', 'request_id': 'req-313b15b8-3cf3-4026-94f7-fb893d986819'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.570402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e78e0ac65149109fd8420ca4914df1', 'request_id': 'req-313b15b8-3cf3-4026-94f7-fb893d986819'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.576852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 980/1947] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:04.576852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.581767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed82f41-ba01-4b31-b6f2-65c362a17802 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.582226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] The token cache is being invalidate because project 82e78e0ac65149109fd8420ca4914df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:04.582514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.583173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:04.586474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 968/1948] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:04 2022] DELETE /identity/v3/projects/82e78e0ac65149109fd8420ca4914df1 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:04.590069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.590069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.592148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.592873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.593222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.593586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.595167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:04.595167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.596725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e1d01-3172-45bf-a176-aa0a7f16ce20 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.607220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b15b8-3cf3-4026-94f7-fb893d986819 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.612564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973809dd-f28e-4379-a2d1-944343708973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.613236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973809dd-f28e-4379-a2d1-944343708973 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.613750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973809dd-f28e-4379-a2d1-944343708973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.614255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973809dd-f28e-4379-a2d1-944343708973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.614700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 981/1949] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:04.616834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.617099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.617367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.619667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973809dd-f28e-4379-a2d1-944343708973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.621740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973809dd-f28e-4379-a2d1-944343708973 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.642654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:04.643931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.649599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:04.653733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 982/1950] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:04.654355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.655648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.670446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef3414-ca8d-4031-a1e3-63bffefae67c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.675303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.676596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.682806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.682806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.682806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.682806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.707436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 969/1951] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:04.711417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.711417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.711417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.731454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:04.732832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.812690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 983/1952] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:04.812690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.836412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 970/1953] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:04.837266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:04.838559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:04.854350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5b7ed-953d-452d-90a1-9bdcbb3b1cfb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.922580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09adbd-6745-48db-a1d4-b34e29964572 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:04.927325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.927928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.928226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.929354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.929997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.931936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:04.935822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:04.936494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:04.936780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:04.937159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] PATH_INFO: `/v3/users/777929672d6e43238550b19ffb719e90` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:04.938642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:04.941316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=777929672d6e43238550b19ffb719e90)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:04.948772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.023947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 984/1954] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:04 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:05.023947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.031927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af989b-b4d4-43c8-8aa7-31dbecfd2bdf admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.042787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.042787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.042787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.042787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.043249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.044460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.145972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 985/1955] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:05 2022] POST /identity/v3/users => generated 281 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:05.150260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.154113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '777929672d6e43238550b19ffb719e90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.161659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5ffe2-75e6-41b8-9a90-6f8d6aa51cd6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.168828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.169516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.169871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.170213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.170955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:05.171330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.175774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.190386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 986/1956] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:05.192311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.206768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9413e7-7adf-4e4d-bc70-1f9870900ecd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.212381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.213101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.213379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.213643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.214279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.215880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.226002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 987/1957] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:05.226888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.238752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81918882-6252-4a18-8612-78ff05b7006a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.243716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.244316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.244648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.244919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.245552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.247261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.255576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '777929672d6e43238550b19ffb719e90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.257083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '777929672d6e43238550b19ffb719e90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.263689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 988/1958] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:05.264704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.266237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '777929672d6e43238550b19ffb719e90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.275691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb301d-79ee-4cfb-8060-58bdcd2baed2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.279457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '777929672d6e43238550b19ffb719e90'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:05.280620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c2035-e660-49a6-a067-d800fb96c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.281248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c2035-e660-49a6-a067-d800fb96c98b admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.281526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c2035-e660-49a6-a067-d800fb96c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.281794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c2035-e660-49a6-a067-d800fb96c98b admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.282438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c2035-e660-49a6-a067-d800fb96c98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.284327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c2035-e660-49a6-a067-d800fb96c98b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.286088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 971/1959] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:04 2022] DELETE /identity/v3/users/777929672d6e43238550b19ffb719e90 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:05.286930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.288503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.351868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63767-f597-44c3-8367-a43808c61f62 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.356358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.362874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.362874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.362874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.362874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.362874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.370349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 972/1960] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:05.371144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.372859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.382954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 989/1961] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:05 2022] POST /identity/v3/projects => generated 355 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:05.384362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.384721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.385189 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.396786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dcbd1-f8c0-4daf-843e-76f76691ffb1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.405252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:05.406447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.455497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.456271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.457882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.457882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.457882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.462214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.468499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 990/1962] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:05.469581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.484296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 973/1963] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:05.484296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.488206 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.490121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fadac-eb6a-466b-8efa-efa228fe4635 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.504741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.507217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.507589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.507868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] PATH_INFO: `/v3/projects/95a0373545a14aadb3a31f7e9c3ab175/users/adeec964a47e4da19d7425001490f712/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.519301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b9b4f-22dc-4547-8dba-68656c151629 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.526401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a0373545a14aadb3a31f7e9c3ab175, user_id=adeec964a47e4da19d7425001490f712, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.529343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.545119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.545865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.549184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.549184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.549184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.549184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.626282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:05.672612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 991/1964] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:05 2022] PUT /identity/v3/projects/95a0373545a14aadb3a31f7e9c3ab175/users/adeec964a47e4da19d7425001490f712/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:05.673606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.675333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.682623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 974/1965] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:05 2022] POST /identity/v3/users => generated 278 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:05.684083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.699284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7307f-486b-4c0b-8a22-2ee61c0b9158 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.757472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525301a2-623e-4025-b53e-b10b82fe3bf0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.764782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec6374-da09-449e-ab15-02eee104906e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.765513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec6374-da09-449e-ab15-02eee104906e service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.765861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec6374-da09-449e-ab15-02eee104906e service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.766209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec6374-da09-449e-ab15-02eee104906e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.799098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.799956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.800240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.802564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] PATH_INFO: `/v3/projects/e3093b23c0544570a76df1cd72616b59/users/d2dd6fafa9614fad963eff3715ff41cf/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.820274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3093b23c0544570a76df1cd72616b59, user_id=d2dd6fafa9614fad963eff3715ff41cf, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.823090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.833960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec6374-da09-449e-ab15-02eee104906e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.834268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ec6374-da09-449e-ab15-02eee104906e service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:05.845802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 992/1966] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:05.848842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.853294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.853294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.884872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] MFA Rules not processed for user `adeec964a47e4da19d7425001490f712`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:05.886146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:05.907901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 975/1967] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:05 2022] PUT /identity/v3/projects/e3093b23c0544570a76df1cd72616b59/users/d2dd6fafa9614fad963eff3715ff41cf/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:05.909474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:05.988971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bdda6-dc2e-4b1b-b040-1b953e677be4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:05.995212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52960-186c-48c8-986e-4535083962d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:05.995866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52960-186c-48c8-986e-4535083962d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:05.996145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52960-186c-48c8-986e-4535083962d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:05.996443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52960-186c-48c8-986e-4535083962d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:05.997099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52960-186c-48c8-986e-4535083962d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:05.999008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52960-186c-48c8-986e-4535083962d2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.011211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 976/1968] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:06.012775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.013077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.013349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.030594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 993/1969] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:06.031348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.048231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] MFA Rules not processed for user `d2dd6fafa9614fad963eff3715ff41cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:06.048501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443f0a0-00ea-476c-a57c-9076d424bba8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.049852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.053880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.054107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.058692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.058692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.058692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.058692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.067483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 994/1970] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:06.070529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.073148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.159293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f270-7fa1-4bbb-ab69-e0a94c24d210 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.166299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.166929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.167205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.167476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.193547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 977/1971] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:06.193547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.217609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae326f7-68b4-41e8-91a2-6c265d4b5790 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.223107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.223859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.224261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.224622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] PATH_INFO: `/v3/projects/f8ad9536a62f449a870f920c10b51032/users/1942bd68cc2947b29b88aa9322acce6b/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.236896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.238357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.248128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ad9536a62f449a870f920c10b51032, user_id=1942bd68cc2947b29b88aa9322acce6b, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.251023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 995/1972] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:06.251962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.286462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.297017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abcc27-ee7e-410e-ac5a-6726b7ed8165 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.302176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.302811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.303099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.303648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] PATH_INFO: `/v3/users/adeec964a47e4da19d7425001490f712` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.305108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adeec964a47e4da19d7425001490f712)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.307038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.341779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 978/1973] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:06 2022] PUT /identity/v3/projects/f8ad9536a62f449a870f920c10b51032/users/1942bd68cc2947b29b88aa9322acce6b/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:06.342737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.411498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a5bed-b1d8-4ae1-a2d1-ffd09f98cf07 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.416323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2db3ce-9e16-4a61-a887-e3eddb0ce119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.416976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2db3ce-9e16-4a61-a887-e3eddb0ce119 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.417249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2db3ce-9e16-4a61-a887-e3eddb0ce119 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.417560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2db3ce-9e16-4a61-a887-e3eddb0ce119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.480810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2db3ce-9e16-4a61-a887-e3eddb0ce119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.482383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2db3ce-9e16-4a61-a887-e3eddb0ce119 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.494431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 979/1974] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:06.496145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.496455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.497315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.533021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adeec964a47e4da19d7425001490f712'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.537268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adeec964a47e4da19d7425001490f712'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.537558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] MFA Rules not processed for user `1942bd68cc2947b29b88aa9322acce6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:06.538820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.542113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adeec964a47e4da19d7425001490f712'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.604400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 980/1975] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:06.605252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.606517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.614095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adeec964a47e4da19d7425001490f712'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.614718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adeec964a47e4da19d7425001490f712'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.634050 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 996/1976] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:06 2022] DELETE /identity/v3/users/adeec964a47e4da19d7425001490f712 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:06.635310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:06.636776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:06.678490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a59db6-5582-4935-a1bb-6d4b65699858 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.710222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe2f4b-9aa5-4816-aa42-a9a21abf7520 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:06.714911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.715547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.715826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.716099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] PATH_INFO: `/v3/users/d2dd6fafa9614fad963eff3715ff41cf` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.717276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2dd6fafa9614fad963eff3715ff41cf)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.719200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.735077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:06.735729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:06.736006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:06.736304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] PATH_INFO: `/v3/projects/e178d1bca043413bae28b1103452ccc8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:06.737451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e178d1bca043413bae28b1103452ccc8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:06.739541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:06.780292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:06.921720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e178d1bca043413bae28b1103452ccc8', 'request_id': 'req-0aab4ffa-495a-462a-8616-1741b6218f18'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.929105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2dd6fafa9614fad963eff3715ff41cf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.930170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2dd6fafa9614fad963eff3715ff41cf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:06.945533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2dd6fafa9614fad963eff3715ff41cf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.042773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e178d1bca043413bae28b1103452ccc8', 'request_id': 'req-0aab4ffa-495a-462a-8616-1741b6218f18'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.049905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e178d1bca043413bae28b1103452ccc8', 'request_id': 'req-0aab4ffa-495a-462a-8616-1741b6218f18'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.059344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:07.065277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] The token cache is being invalidate because project e178d1bca043413bae28b1103452ccc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:07.065726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.069076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.073462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 981/1977] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:06 2022] DELETE /identity/v3/projects/e178d1bca043413bae28b1103452ccc8 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:07.073462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.073462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.107599 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2dd6fafa9614fad963eff3715ff41cf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.108263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebae5819-47d5-48ee-b523-3bae17ca8ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2dd6fafa9614fad963eff3715ff41cf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.130276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 997/1978] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:06 2022] DELETE /identity/v3/users/d2dd6fafa9614fad963eff3715ff41cf => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:07.132109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.132469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.132810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.157511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:07.158643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.179952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab4ffa-495a-462a-8616-1741b6218f18 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.235386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 998/1979] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:07.236132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.237553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.248864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42be1b8-1975-4b8d-bce3-33579f47bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.249591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be1b8-1975-4b8d-bce3-33579f47bd55 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.249978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be1b8-1975-4b8d-bce3-33579f47bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.250309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42be1b8-1975-4b8d-bce3-33579f47bd55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.250589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ddde0-6340-42ac-8995-5c4398b9d34e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.251185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42be1b8-1975-4b8d-bce3-33579f47bd55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.253797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42be1b8-1975-4b8d-bce3-33579f47bd55 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.255573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.256653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.257039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.257434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] PATH_INFO: `/v3/users/7f5d03bc79474fd2a519baaeae22a5d5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.258865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5d03bc79474fd2a519baaeae22a5d5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.260961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.375344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 982/1980] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:07 2022] POST /identity/v3/projects => generated 355 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:07.376911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.377351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.377627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.399187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:07.400617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.467622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 983/1981] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:07.468652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.470359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.494161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5d03bc79474fd2a519baaeae22a5d5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.497961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5d03bc79474fd2a519baaeae22a5d5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.504645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5d03bc79474fd2a519baaeae22a5d5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.532263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdba743-29a8-46f7-9599-d36cab90624b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.536234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.536904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.537176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.538029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.547886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.549377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.560480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 984/1982] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:07.561518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.562955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.574821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5d03bc79474fd2a519baaeae22a5d5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.578381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5d03bc79474fd2a519baaeae22a5d5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:07.586330 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e93881-5354-4a1c-a0a0-684743e4363a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.591641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.592359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.592683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.593097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.593848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.596368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.602809 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 999/1983] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:07 2022] DELETE /identity/v3/users/7f5d03bc79474fd2a519baaeae22a5d5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:07.603639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.613539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.679940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71db985-c608-455b-a1f3-1ae6b375494d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.683578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.684186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.684493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.684765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.686475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.722701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 985/1984] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:07 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:07.727093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.729488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:07.747603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0492-766b-4f91-adaa-6c0780a7289c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.760421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.761912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.772988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1000/1985] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:07.832316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.833359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.833359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.833660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.834159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.835657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:07.837269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.852425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 986/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:07.853681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:07.863642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245cc59-5236-4fd0-9f98-d22d87726930 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:07.871655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:07.871655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:07.871655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:07.871655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] PATH_INFO: `/v3/projects/409cac5991764678821fc9b91a44cc40/users/6f3b68d5f5f448759eff74fe78343471/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:07.890023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] RBAC: Authorizing `identity:create_grant(project_id=409cac5991764678821fc9b91a44cc40, user_id=6f3b68d5f5f448759eff74fe78343471, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:07.894039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:07.982883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1001/1987] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:07 2022] PUT /identity/v3/projects/409cac5991764678821fc9b91a44cc40/users/6f3b68d5f5f448759eff74fe78343471/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:07.987039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:08.000222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bff87-d4a2-4ca2-ae7f-c8b12682ca7c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:08.046446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:08.049060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:08.092140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:08.092811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.093047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.093324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.094032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:08.095858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:08.112262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 987/1988] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:08.124323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:08.138686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad48390-2c79-4abe-ac9d-a660a8ad97b6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:08.144636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:08.145498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.145917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.146338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] PATH_INFO: `/v3/projects/409cac5991764678821fc9b91a44cc40/users/6f3b68d5f5f448759eff74fe78343471/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.150040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409cac5991764678821fc9b91a44cc40, user_id=6f3b68d5f5f448759eff74fe78343471, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:08.153143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:08.176900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c19f74-ae96-4bf7-86e2-4b70b8c5903e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:08.181147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 988/1989] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:08 2022] PUT /identity/v3/projects/409cac5991764678821fc9b91a44cc40/users/6f3b68d5f5f448759eff74fe78343471/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:08.182559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:08.187618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:08.268219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b32dde-f256-4afd-bc25-efcff5745780 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:08.274487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:08.275759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.276181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.276783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] PATH_INFO: `/v3/projects/d722b580c239432dafef6b3605e89e53` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.278566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:08.279803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d722b580c239432dafef6b3605e89e53)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:08.281033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:08.285392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.285392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.285582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:08.285670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] PATH_INFO: `/v3/projects/95a0373545a14aadb3a31f7e9c3ab175` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.285670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a0373545a14aadb3a31f7e9c3ab175)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:08.285917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:08.399367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd722b580c239432dafef6b3605e89e53', 'request_id': 'req-2d3a4174-71d7-461d-9a83-1932b808a6cd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.419615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:08.429142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a0373545a14aadb3a31f7e9c3ab175', 'request_id': 'req-99075004-8456-4d46-a0a2-84df10135270'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.435274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a0373545a14aadb3a31f7e9c3ab175', 'request_id': 'req-99075004-8456-4d46-a0a2-84df10135270'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.477531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd722b580c239432dafef6b3605e89e53', 'request_id': 'req-2d3a4174-71d7-461d-9a83-1932b808a6cd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.486086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd722b580c239432dafef6b3605e89e53', 'request_id': 'req-2d3a4174-71d7-461d-9a83-1932b808a6cd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.510426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] The token cache is being invalidate because project d722b580c239432dafef6b3605e89e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:08.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2d3a4174-71d7-461d-9a83-1932b808a6cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 989/1990] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:08 2022] DELETE /identity/v3/projects/d722b580c239432dafef6b3605e89e53 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:08.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.518166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.550535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] MFA Rules not processed for user `6f3b68d5f5f448759eff74fe78343471`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:08.551777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:08.556935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a0373545a14aadb3a31f7e9c3ab175', 'request_id': 'req-99075004-8456-4d46-a0a2-84df10135270'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.575554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] The token cache is being invalidate because project 95a0373545a14aadb3a31f7e9c3ab175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:08.575937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.576687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.581411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1002/1991] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:08 2022] DELETE /identity/v3/projects/95a0373545a14aadb3a31f7e9c3ab175 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:08.582302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:08.594173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:08.650519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 990/1992] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:08.651442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:08.652866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:08.680362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99075004-8456-4d46-a0a2-84df10135270 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:08.685748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:08.686384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.686729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.687003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] PATH_INFO: `/v3/projects/e3093b23c0544570a76df1cd72616b59` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.688121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3093b23c0544570a76df1cd72616b59)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:08.690067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:08.726720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d572738-1316-4154-aa46-a3029e4fb4b5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:08.732120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:08.732930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.733278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.733656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] PATH_INFO: `/v3/users/d41b1b2400fa48b99fb8681b55fd07ea` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.734906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41b1b2400fa48b99fb8681b55fd07ea)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:08.737127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:08.822284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3093b23c0544570a76df1cd72616b59', 'request_id': 'req-e3466957-f80d-428b-855e-89a05029425c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.828456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3093b23c0544570a76df1cd72616b59', 'request_id': 'req-e3466957-f80d-428b-855e-89a05029425c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.935299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3093b23c0544570a76df1cd72616b59', 'request_id': 'req-e3466957-f80d-428b-855e-89a05029425c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.947225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] The token cache is being invalidate because project e3093b23c0544570a76df1cd72616b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:08.947511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.947888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41b1b2400fa48b99fb8681b55fd07ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.948177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e3466957-f80d-428b-855e-89a05029425c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:08.949745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1003/1993] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:08 2022] DELETE /identity/v3/projects/e3093b23c0544570a76df1cd72616b59 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:08.950940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:08.951221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:08.951495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:08.970585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:08.971791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.033595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1004/1994] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:09.034509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.035825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.093535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c90b4-839d-4832-a4c6-cc4978a34fef None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.097161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.097795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.098070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.098339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.100893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.123129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41b1b2400fa48b99fb8681b55fd07ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:09.124045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41b1b2400fa48b99fb8681b55fd07ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:09.135471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:09.144809 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41b1b2400fa48b99fb8681b55fd07ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:09.158208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb6501a5-0164-4f03-9181-eeaccca10a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41b1b2400fa48b99fb8681b55fd07ea'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:09.168750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 991/1995] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:08 2022] DELETE /identity/v3/users/d41b1b2400fa48b99fb8681b55fd07ea => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:09.170515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf279d-f61c-4db5-bcca-76cf65e7b591 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.170802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf279d-f61c-4db5-bcca-76cf65e7b591 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.173433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf279d-f61c-4db5-bcca-76cf65e7b591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.180584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.182111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef635349-7898-4132-95f4-267ef9941b6c service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.192749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1005/1996] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:09.194213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.194588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.194957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.204221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-62bf279d-f61c-4db5-bcca-76cf65e7b591 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:09.206786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62bf279d-f61c-4db5-bcca-76cf65e7b591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.217092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:09.226927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.286970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 992/1997] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:09.294995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.295423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.295806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.299591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1006/1998] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:09.304514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.313361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca80e44-7392-4990-8cd6-907880bfa864 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.316162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:09.317489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.318304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.321011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.321011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.321011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.321011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:09.321370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.322899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.327626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 993/1999] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:09.331767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.336463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1007/2000] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:45:09.340569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.344864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b58e6-31a8-49eb-84b6-2486e982dfb8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.349574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.350899 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.351175 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.351289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a33a4-8ecb-4d70-8ae5-68712ca5de7a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.351438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.352084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:09.352409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.354031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.356183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.356812 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.357096 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.357372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.357997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.359889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.365533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 994/2001] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:45:09.370973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.382105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef833-bc66-4db8-9304-0bf4bd7ebc4e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.386836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.387446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.387728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.388001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.388664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.390583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.431951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:09.497238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1008/2002] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:09.501380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.557157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 995/2003] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:09.564518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.586076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39228d9c-4723-4004-8a95-14b2590df19d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.647454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d604e-714d-465b-b4a0-0633db872bf3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.654581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.655223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.655508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.655782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.656996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.667192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.692186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.693223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.693547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.694009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.694656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.696585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.775526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 996/2004] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:09.775526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.776027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1009/2005] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:09 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:09.776396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.778053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.789036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4270a-6704-4abd-abfa-dee0dfebc9b6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.794789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.795398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.795675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.796642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.802151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.804995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.817206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1010/2006] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:09.818191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.820857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.836928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b000f-108c-4e70-ae78-fcb9cb28d275 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.843008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.844838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.845151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.846496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] PATH_INFO: `/v3/projects/b19761659a064e23acfd69163d5925af` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.847805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19761659a064e23acfd69163d5925af)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.850932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.853515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb75d9-34bb-4868-9e10-91bfa13b548e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.857683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:09.858946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:09.859298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:09.861554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:09.861736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:09.904500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:09.936809 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:09.938304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:09.948126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 997/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:09.950021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:09.979956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae6e74-f8cd-4be5-93da-20762d48554b service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:09.993744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19761659a064e23acfd69163d5925af', 'request_id': 'req-e4019887-4cef-4216-ad3a-8fd33c9fdbba'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.002899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19761659a064e23acfd69163d5925af', 'request_id': 'req-e4019887-4cef-4216-ad3a-8fd33c9fdbba'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.045614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.046384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.046784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.047074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.047871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.049865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.061906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 998/2008] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:10.062768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.071814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19761659a064e23acfd69163d5925af', 'request_id': 'req-e4019887-4cef-4216-ad3a-8fd33c9fdbba'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.084844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c2157-2fca-4e2b-b56f-fce157d35c66 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.085156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] The token cache is being invalidate because project b19761659a064e23acfd69163d5925af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:10.085276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.086074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:10.087974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1011/2009] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:09 2022] DELETE /identity/v3/projects/b19761659a064e23acfd69163d5925af => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:10.088510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.096788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.105204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d85d63-88c7-40e5-bcc6-d17ca87352df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.114516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d85d63-88c7-40e5-bcc6-d17ca87352df admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.114516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d85d63-88c7-40e5-bcc6-d17ca87352df admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.114516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d85d63-88c7-40e5-bcc6-d17ca87352df admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.114516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d85d63-88c7-40e5-bcc6-d17ca87352df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.114857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4019887-4cef-4216-ad3a-8fd33c9fdbba admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.114952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d85d63-88c7-40e5-bcc6-d17ca87352df admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.122863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.123762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.124047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.124321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.125053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.126764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.127365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 999/2010] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:10.128695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.130666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.130666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.143180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1012/2011] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:10.145517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.149311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.151728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:10.152945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.162661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533bea2-d8fc-4bc2-ad07-e0b6ad836adb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.163931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1000/2012] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:10.164777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.175762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d3786-508e-4ab1-9009-ee70f68770cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.179669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d3786-508e-4ab1-9009-ee70f68770cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.180255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d3786-508e-4ab1-9009-ee70f68770cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.180731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a85a86-44fa-41a0-b10d-c084eaee5607 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.180984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d3786-508e-4ab1-9009-ee70f68770cb admin admin] PATH_INFO: `/v3/projects/b1017dfe87f64ef69cad719ccabda993/users/4abc64377cf54df1b2da448fc3bae463/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.192624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc7084-5c15-46d7-8c7d-5543bb2da515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.194149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc7084-5c15-46d7-8c7d-5543bb2da515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.194430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc7084-5c15-46d7-8c7d-5543bb2da515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.194707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc7084-5c15-46d7-8c7d-5543bb2da515 admin admin] PATH_INFO: `/v3/projects/ebdf8c73c38f40238e94d414804504ca/users/6d67f93662e348f49ba953383a3ac645/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.201476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:10.211565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d3786-508e-4ab1-9009-ee70f68770cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1017dfe87f64ef69cad719ccabda993, user_id=4abc64377cf54df1b2da448fc3bae463, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.212303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc7084-5c15-46d7-8c7d-5543bb2da515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebdf8c73c38f40238e94d414804504ca, user_id=6d67f93662e348f49ba953383a3ac645, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.214865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d3786-508e-4ab1-9009-ee70f68770cb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.215062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc7084-5c15-46d7-8c7d-5543bb2da515 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.284141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1001/2013] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:10 2022] PUT /identity/v3/projects/ebdf8c73c38f40238e94d414804504ca/users/6d67f93662e348f49ba953383a3ac645/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:10.285816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07534fa4-a3c1-4247-a505-1e5dd92f110e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.285816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07534fa4-a3c1-4247-a505-1e5dd92f110e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.285986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07534fa4-a3c1-4247-a505-1e5dd92f110e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.300639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1013/2014] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:10 2022] PUT /identity/v3/projects/b1017dfe87f64ef69cad719ccabda993/users/4abc64377cf54df1b2da448fc3bae463/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:10.303981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.304566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.305177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.329652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-07534fa4-a3c1-4247-a505-1e5dd92f110e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:10.330868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07534fa4-a3c1-4247-a505-1e5dd92f110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.352293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] MFA Rules not processed for user `6d67f93662e348f49ba953383a3ac645`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:10.355071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.403090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1002/2015] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:10.407759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.408045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.408317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.446580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] MFA Rules not processed for user `4abc64377cf54df1b2da448fc3bae463`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:10.446580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.460677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1014/2016] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:10.461551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.476233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1d51b-dee5-42e1-9b1c-ab82547b7be1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.481699 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.482315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.482598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.482865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.483663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:10.484019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.486202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.503082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1015/2017] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:10.512421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.526210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3e9fd-cff2-4339-a091-9d0a19f2ee04 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.533286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.534403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.535123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.535570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.536503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.541015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.604541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1003/2018] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:10.606986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.608441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.678771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1016/2019] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:10.678771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.681238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:10.685085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a20959-29bc-4613-a6e5-72480ab09a2e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.761420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.762970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.763536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.763814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.786744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddff1dd-d4f3-48f6-b92a-e1b2806d0190 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.792847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.793575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.794002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.794287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.846204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.848044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.861932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1004/2020] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:10.862611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:10.952257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.958683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.963004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbe4ec-9ac2-4cb9-8765-25d4f6d433e3 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:10.969218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:10.969756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:10.970132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:10.970507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:10.971248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:10.973936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:10.989900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1017/2021] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:11.065949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:11.066967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1005/2022] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:10 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:11.077101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.102417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d4156-e9ed-4486-8b5e-5a600955015f service glance] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.109541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.110465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.110895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.111314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.112212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.114227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.143870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1018/2023] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:11.148143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.156585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:11.165800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0afab2-6255-477b-ae46-aa3566196ed0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.177652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e530-70ba-4f1b-aebe-aebd70c49e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.179162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e530-70ba-4f1b-aebe-aebd70c49e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.179553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e530-70ba-4f1b-aebe-aebd70c49e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.179835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e530-70ba-4f1b-aebe-aebd70c49e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.180543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1e530-70ba-4f1b-aebe-aebd70c49e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.184684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1e530-70ba-4f1b-aebe-aebd70c49e90 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.222114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1006/2024] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:11.227024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.238641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea5ad-2de0-4f68-969e-a04264023abc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.245303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.246370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.246892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.247399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] PATH_INFO: `/v3/projects/db565314dd8c4d6cbe65c13dbea4cdc4/users/6f1c266a87c147d598bf327b56556852/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.268864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db565314dd8c4d6cbe65c13dbea4cdc4, user_id=6f1c266a87c147d598bf327b56556852, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.273472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.411252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.411786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.412267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.430493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1019/2025] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:11 2022] PUT /identity/v3/projects/db565314dd8c4d6cbe65c13dbea4cdc4/users/6f1c266a87c147d598bf327b56556852/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:11.465609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] MFA Rules not processed for user `6f1c266a87c147d598bf327b56556852`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:11.467104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:11.559728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1007/2026] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:11.569617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:11.649246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df12706-6313-4ce6-bab9-f7e608c807c6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:11.661576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4fc8c7-c7e4-4deb-8275-ea1d5c42479c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:11.662253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fc8c7-c7e4-4deb-8275-ea1d5c42479c service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:11.662538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fc8c7-c7e4-4deb-8275-ea1d5c42479c service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:11.662817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4fc8c7-c7e4-4deb-8275-ea1d5c42479c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:11.679941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fc8c7-c7e4-4deb-8275-ea1d5c42479c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:11.681643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4fc8c7-c7e4-4deb-8275-ea1d5c42479c service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:11.691396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1020/2027] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:11.860678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:12.578459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:12.695647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:14.745664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:14.853411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:15.484135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:15.554545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453f193-85a5-476f-add1-3c9e89610796 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:15.561238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:15.562185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:15.562563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:15.562979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] PATH_INFO: `/v3/users/6d67f93662e348f49ba953383a3ac645` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:15.564403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d67f93662e348f49ba953383a3ac645)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:15.567713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:15.889703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d67f93662e348f49ba953383a3ac645'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.979943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d67f93662e348f49ba953383a3ac645'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.980941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d67f93662e348f49ba953383a3ac645'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:15.991373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d67f93662e348f49ba953383a3ac645'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:16.004967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d67f93662e348f49ba953383a3ac645'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:16.012770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1008/2028] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:15 2022] DELETE /identity/v3/users/6d67f93662e348f49ba953383a3ac645 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:16.018627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.019188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.019282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.045322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:16.048356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:16.049306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:16.077934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:16.124271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1021/2029] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:16.134767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:16.136331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:16.199051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac543a-43f4-4123-8c76-d5003759adb6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:16.202576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:16.203185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.203466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.203739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.213120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:16.214572 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.224208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1009/2030] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:16.715960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:16.717452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:16.734714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33d882-32d2-4ced-933d-4532f6ee2e4c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:16.742033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:16.749194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:16.749516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:16.749769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] PATH_INFO: `/v3/projects/ebdf8c73c38f40238e94d414804504ca` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:16.753250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebdf8c73c38f40238e94d414804504ca)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:16.767558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:16.910389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebdf8c73c38f40238e94d414804504ca', 'request_id': 'req-55681090-8f9b-4433-b8d3-57874b1c251c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:16.917465 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebdf8c73c38f40238e94d414804504ca', 'request_id': 'req-55681090-8f9b-4433-b8d3-57874b1c251c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:16.996413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebdf8c73c38f40238e94d414804504ca', 'request_id': 'req-55681090-8f9b-4433-b8d3-57874b1c251c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:17.009051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.010641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] The token cache is being invalidate because project ebdf8c73c38f40238e94d414804504ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:17.010985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:17.011634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-55681090-8f9b-4433-b8d3-57874b1c251c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:17.013326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1022/2031] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:16 2022] DELETE /identity/v3/projects/ebdf8c73c38f40238e94d414804504ca => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:17.111335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.433758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.448647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.512056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.529014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:INSERT=7,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.665393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.779459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:17.871678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.105339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.168121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.328638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.351821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.431476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.504566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.578265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.608138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:18.934180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:19.014162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:19.449426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:19.567552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:19.888209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:19.957526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.105710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.151062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.179433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.208214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.481702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.517452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.682585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.710845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.883713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:20.945339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:21.517470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:21.624306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:22.199557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:22.419533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:22.807216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:23.235249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:23.598253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:24.426846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:30.485586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:30.487295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:30.501335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75a46d-9640-431f-80db-95bf3d34b069 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:30.562636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:30.563294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:30.563579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:30.563853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] PATH_INFO: `/v3/users/1942bd68cc2947b29b88aa9322acce6b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:30.565060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1942bd68cc2947b29b88aa9322acce6b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:30.567114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:30.694584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:30.798269 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1942bd68cc2947b29b88aa9322acce6b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:30.904545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1942bd68cc2947b29b88aa9322acce6b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:30.905184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1942bd68cc2947b29b88aa9322acce6b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:30.923462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1942bd68cc2947b29b88aa9322acce6b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:30.944902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1942bd68cc2947b29b88aa9322acce6b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:30.951025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1010/2032] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:30 2022] DELETE /identity/v3/users/1942bd68cc2947b29b88aa9322acce6b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:30.955482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:30.955972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:30.956072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:30.975519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:30.976765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:31.038040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1023/2033] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:31.048700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:31.050544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:31.113313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ed776-36ba-4bb8-a728-9daf3befc085 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:31.118015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:31.118632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:31.118947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:31.119170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:31.128510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:31.131250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:31.141117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1011/2034] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:31.719870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:31.721514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:31.735520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0fb88-dc54-41c9-b25f-f5422349b599 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:31.750618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:31.751300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:31.751586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:31.751876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] PATH_INFO: `/v3/projects/f8ad9536a62f449a870f920c10b51032` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:31.753482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ad9536a62f449a870f920c10b51032)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:31.758973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:31.948701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ad9536a62f449a870f920c10b51032', 'request_id': 'req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:31.956900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ad9536a62f449a870f920c10b51032', 'request_id': 'req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.020720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ad9536a62f449a870f920c10b51032', 'request_id': 'req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.035787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] The token cache is being invalidate because project f8ad9536a62f449a870f920c10b51032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:32.036416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.038155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.040200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1024/2035] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:31 2022] DELETE /identity/v3/projects/f8ad9536a62f449a870f920c10b51032 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:32.043974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:32.055497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73589-f464-4b33-a421-2a4e91cfe89d service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:32.131596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:32.132256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:32.132555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:32.132826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] PATH_INFO: `/v3/users/6f3b68d5f5f448759eff74fe78343471` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:32.133993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3b68d5f5f448759eff74fe78343471)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:32.135994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:32.296583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3b68d5f5f448759eff74fe78343471'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.298457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:32.396153 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3b68d5f5f448759eff74fe78343471'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.396807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3b68d5f5f448759eff74fe78343471'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.418603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3b68d5f5f448759eff74fe78343471'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.442261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0680de07-99a3-4d15-9dc0-62020168550e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3b68d5f5f448759eff74fe78343471'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:32.444207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1012/2036] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:32 2022] DELETE /identity/v3/users/6f3b68d5f5f448759eff74fe78343471 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:33.136583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:33.138271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:33.153910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480bea0-9fc3-43e1-ac4a-b390eb147b9b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:33.219274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:33.219989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.220427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.220857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] PATH_INFO: `/v3/projects/409cac5991764678821fc9b91a44cc40` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.222526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=409cac5991764678821fc9b91a44cc40)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:33.225119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:33.226821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:33.334576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409cac5991764678821fc9b91a44cc40', 'request_id': 'req-0e915a01-145f-4c00-a896-f2e12b87d305'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.340898 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409cac5991764678821fc9b91a44cc40', 'request_id': 'req-0e915a01-145f-4c00-a896-f2e12b87d305'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.447076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409cac5991764678821fc9b91a44cc40', 'request_id': 'req-0e915a01-145f-4c00-a896-f2e12b87d305'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.466845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] The token cache is being invalidate because project 409cac5991764678821fc9b91a44cc40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:33.467333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.468195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0e915a01-145f-4c00-a896-f2e12b87d305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:33.473648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1025/2037] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:33 2022] DELETE /identity/v3/projects/409cac5991764678821fc9b91a44cc40 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:33.475462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.475937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.476017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.499211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:33.500362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:33.569983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1013/2038] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:33.577798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.578084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.578357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.599904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:33.601155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:33.610228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1026/2039] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:33.615071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:33.625086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf85f7-2d72-43b1-8ecc-f5b2b1a49d3e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:33.629769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:33.630642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.630921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.631191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.631844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:33.632217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:33.633880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:33.646259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1014/2040] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:45:33.650689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:33.659890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ee4fe-6c2c-445a-ac73-ec702baeedb7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:33.665374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:33.665971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.666249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.666522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.667180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:33.669282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:33.751376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1027/2041] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:33 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:33.756329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:33.766785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316227f-b9ff-4900-ae7b-ba6d79b9b1e3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:33.880927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:33.881549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:33.881870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:33.882101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:33.882749 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:33.897949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.027148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1015/2042] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:33 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:34.035233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.051839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917386-298f-4ac6-bc82-4cb0ea944438 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.057360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.057971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.058242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.058561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.059220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.060911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.072612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1028/2043] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:34.076840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.089490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912dbab-20a5-4760-988f-885c1cacbf13 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.095486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34684f2-fbd6-4813-ade8-7dc3f223d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.096115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34684f2-fbd6-4813-ade8-7dc3f223d1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.096414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34684f2-fbd6-4813-ade8-7dc3f223d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.096683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34684f2-fbd6-4813-ade8-7dc3f223d1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.097336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34684f2-fbd6-4813-ade8-7dc3f223d1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.099023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34684f2-fbd6-4813-ade8-7dc3f223d1ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.115926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1016/2044] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:34.119992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.132080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecf747-7863-4227-8bcb-aef0a13ec012 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.154304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.155056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.155393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.155729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] PATH_INFO: `/v3/projects/b165814e28874a898546629aa4d55fad/users/234f936cc2d141339855a3349b5994f3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.177892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b165814e28874a898546629aa4d55fad, user_id=234f936cc2d141339855a3349b5994f3, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.178733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.263412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1029/2045] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:34 2022] PUT /identity/v3/projects/b165814e28874a898546629aa4d55fad/users/234f936cc2d141339855a3349b5994f3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:34.282820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.283365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.283802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.326311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] MFA Rules not processed for user `234f936cc2d141339855a3349b5994f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:34.326311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:34.346858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:34.402208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1017/2046] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:34.403710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.489296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab94646-6ce2-4d52-a527-77980a7fb9cd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.495728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fd987-bd09-445d-97b2-10d354a32220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.496371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fd987-bd09-445d-97b2-10d354a32220 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.496662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fd987-bd09-445d-97b2-10d354a32220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.496941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fd987-bd09-445d-97b2-10d354a32220 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.497660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fd987-bd09-445d-97b2-10d354a32220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.499737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fd987-bd09-445d-97b2-10d354a32220 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.604670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1030/2047] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:34.609057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.622370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d21535-9930-42f7-98fa-20d7d13963a7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.681143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.681802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.682082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.682349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.682970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.684938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.763106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69acd8-88ba-4dc8-82e4-28e143984f15 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.763400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69acd8-88ba-4dc8-82e4-28e143984f15 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.763679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69acd8-88ba-4dc8-82e4-28e143984f15 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.774240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1031/2048] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:45:34.774881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.775193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.775441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.795634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1018/2049] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:34.799695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.800632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:34.802020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:34.808850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:34.813814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203dbab-12b1-452e-b98e-b1739bcc3408 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.823948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.823948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.823948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.823948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.833051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.834812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.848853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1019/2050] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:34.853143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.864988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79974fb-7faf-4ec1-afc7-112a19e25da6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.870021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.870693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.870979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.871298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] PATH_INFO: `/v3/projects/08b331282e5c4282921c747a05f3dd18/users/999c5e5ed9fd4c2082ef4afbaa6af768/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.909896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b331282e5c4282921c747a05f3dd18, user_id=999c5e5ed9fd4c2082ef4afbaa6af768, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.913315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.914891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1032/2051] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:34.917903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.919339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:34.939132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7b5f8-4ee7-4327-806a-df94a16686ab None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:34.944087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:34.944824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:34.945098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:34.945364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] PATH_INFO: `/v3/users/491e2b9a44bd47b190b1a0b1250bf1f3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:34.946494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=491e2b9a44bd47b190b1a0b1250bf1f3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:34.948587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:34.984101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1020/2052] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:34 2022] PUT /identity/v3/projects/08b331282e5c4282921c747a05f3dd18/users/999c5e5ed9fd4c2082ef4afbaa6af768/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:34.985106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:34.986768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.059007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c545e91-b5f9-43d8-b3a4-efda41bca757 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.064363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.065004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.065849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.065849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.075414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.077059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.088822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1021/2053] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:35.093385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.095165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:35.103950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491e2b9a44bd47b190b1a0b1250bf1f3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.105360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491e2b9a44bd47b190b1a0b1250bf1f3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.122079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491e2b9a44bd47b190b1a0b1250bf1f3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.162663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4827-63c2-4aae-a9c1-4b9bd65e065c service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.167559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.168172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.168485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.169236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.169523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.171780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.183275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1022/2054] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:35.187374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.189976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491e2b9a44bd47b190b1a0b1250bf1f3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.190823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dff469e8-305a-4993-9f78-1e8671941941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491e2b9a44bd47b190b1a0b1250bf1f3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.199108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447dbd1-b9ff-42d7-8ccb-de435b59a218 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.205730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.206701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.207819 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.208772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] PATH_INFO: `/v3/projects/08b331282e5c4282921c747a05f3dd18/users/999c5e5ed9fd4c2082ef4afbaa6af768/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.211217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b331282e5c4282921c747a05f3dd18, user_id=999c5e5ed9fd4c2082ef4afbaa6af768, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.214065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.221621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1033/2055] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:34 2022] DELETE /identity/v3/users/491e2b9a44bd47b190b1a0b1250bf1f3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:35.225633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1023/2056] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:35 2022] PUT /identity/v3/projects/08b331282e5c4282921c747a05f3dd18/users/999c5e5ed9fd4c2082ef4afbaa6af768/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:35.236618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.236899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.237167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.266519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] MFA Rules not processed for user `999c5e5ed9fd4c2082ef4afbaa6af768`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:35.267821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.345359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1034/2057] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:35.356686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.358533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.419647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80175d-1972-4e42-9e94-9eda5d8511f1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.469552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.470092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.470363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.470627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.480257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.481731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.488010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.493098 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.493217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1024/2058] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:35.507250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:35.596987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11038e-8626-4f9d-994f-53eab8c6ff30 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.603400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.604036 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.604308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.604619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] PATH_INFO: `/v3/projects/d09164dc6ad44a87b7fae777d9f8435a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.605757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09164dc6ad44a87b7fae777d9f8435a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.607655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.634074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.642862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fefa49-74e0-4cb7-95ce-4da7567499c5 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.647174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.647784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.648054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.648319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.650096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.695958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:35.711685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.712759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09164dc6ad44a87b7fae777d9f8435a', 'request_id': 'req-0275b90a-a378-4382-b25b-0f062e26290f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.713442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6b57a-3142-46cf-b89f-addd43e0d558 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.719069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09164dc6ad44a87b7fae777d9f8435a', 'request_id': 'req-0275b90a-a378-4382-b25b-0f062e26290f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.723905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1025/2059] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:35.783667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09164dc6ad44a87b7fae777d9f8435a', 'request_id': 'req-0275b90a-a378-4382-b25b-0f062e26290f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.795941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] The token cache is being invalidate because project d09164dc6ad44a87b7fae777d9f8435a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:35.796282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.796972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0275b90a-a378-4382-b25b-0f062e26290f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:35.798741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1035/2060] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:35 2022] DELETE /identity/v3/projects/d09164dc6ad44a87b7fae777d9f8435a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:45:35.805180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.805677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.806146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.828262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:35.831784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.901235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1026/2061] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:35.909040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.909308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.909579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.931653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:35.936985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:35.941885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1036/2062] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:35.946275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.959021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03329bb5-ed24-4e7f-ab1d-c6c8f26c1cb1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:35.963619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:35.964251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:35.964567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:35.964838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:35.965487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:35.965788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:35.967401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:35.979671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1027/2063] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:35.984097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:35.996071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64776d2e-3bf9-412b-9782-7a374745ead6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:36.000935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:36.001534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.001808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.002075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.002713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:36.004810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:36.122761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1037/2064] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:35 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:36.127562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:36.138357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea1e26-83a6-4fbd-8a35-4015539a45c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:36.196594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:36.197193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.197468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.197737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.198362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:36.200381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:36.310177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1028/2065] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:36 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:36.314376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:36.325425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c120-da5e-4135-a470-0f550f5a38b6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:36.330199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:36.331494 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.339602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.339884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.341633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:36.343407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:36.355431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1038/2066] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:36.359666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:36.372565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812758c1-1760-4844-ba6a-ede99fe5993f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:36.377730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113950b-aa82-4754-9d56-d811f756eb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:36.378461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113950b-aa82-4754-9d56-d811f756eb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.378839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113950b-aa82-4754-9d56-d811f756eb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.379301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113950b-aa82-4754-9d56-d811f756eb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.380447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113950b-aa82-4754-9d56-d811f756eb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:36.382210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113950b-aa82-4754-9d56-d811f756eb8b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:36.393622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1029/2067] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:36.404332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:36.416627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e47ed3-1fc3-4198-bb7c-907e67f4fb95 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:36.423332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:36.424151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.424459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.424730 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] PATH_INFO: `/v3/projects/52f30f320fa04cdfa0e1d4ea9050119c/users/fc27435df46542a3ac37e743b6399387/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.450546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f30f320fa04cdfa0e1d4ea9050119c, user_id=fc27435df46542a3ac37e743b6399387, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:36.453278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:36.515839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1039/2068] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:36 2022] PUT /identity/v3/projects/52f30f320fa04cdfa0e1d4ea9050119c/users/fc27435df46542a3ac37e743b6399387/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:36.533271 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.533565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.533840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.554360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:36.556497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:36.602542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:36.617111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1030/2069] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:36.626925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:36.629410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:36.670035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:36.693830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ffa67-76f8-4b14-8bfa-e3a343da04d4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:36.697688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86085d-d23c-4bdf-b00a-f94fc084f248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:36.698293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86085d-d23c-4bdf-b00a-f94fc084f248 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:36.698570 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86085d-d23c-4bdf-b00a-f94fc084f248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:36.698840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86085d-d23c-4bdf-b00a-f94fc084f248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:36.709801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86085d-d23c-4bdf-b00a-f94fc084f248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:36.711239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86085d-d23c-4bdf-b00a-f94fc084f248 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:36.721337 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1040/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:38.907692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:38.909229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:38.921835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532eccbd-7263-4274-95b1-c6bc605bc768 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:38.926747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:38.929195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:38.929195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:38.929195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] PATH_INFO: `/v3/users/234f936cc2d141339855a3349b5994f3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:38.929195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=234f936cc2d141339855a3349b5994f3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:38.931398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:39.164250 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234f936cc2d141339855a3349b5994f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:39.230254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234f936cc2d141339855a3349b5994f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:39.230489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234f936cc2d141339855a3349b5994f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:39.249579 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234f936cc2d141339855a3349b5994f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:39.275604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234f936cc2d141339855a3349b5994f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:39.286686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1031/2071] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:38 2022] DELETE /identity/v3/users/234f936cc2d141339855a3349b5994f3 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:39.288487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:39.288942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:39.289165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:39.310252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:39.311460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:39.383544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1041/2072] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:39.393158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:39.394916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:39.466547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946baec-2e5f-4392-95ec-b8595ab8f170 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:39.470347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:39.470973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:39.471248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:39.471538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:39.482741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:39.484208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:39.495664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1032/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:40.136468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:40.138144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:40.154976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95c51-1eb1-4ce4-95b3-17f97a32ae81 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:40.158824 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:40.159867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:40.160573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:40.161061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] PATH_INFO: `/v3/projects/b165814e28874a898546629aa4d55fad` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:40.162620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b165814e28874a898546629aa4d55fad)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:40.167243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:40.296644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b165814e28874a898546629aa4d55fad', 'request_id': 'req-ae24a633-d22d-4769-b16c-80499493d792'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.304954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b165814e28874a898546629aa4d55fad', 'request_id': 'req-ae24a633-d22d-4769-b16c-80499493d792'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.378717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b165814e28874a898546629aa4d55fad', 'request_id': 'req-ae24a633-d22d-4769-b16c-80499493d792'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.398422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] The token cache is being invalidate because project b165814e28874a898546629aa4d55fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:40.398743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.399410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.404702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1042/2074] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:40 2022] DELETE /identity/v3/projects/b165814e28874a898546629aa4d55fad => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:40.406163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:40.417980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30430fff-12b3-4744-bf3f-ad0ac2d58f6f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:40.504473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:40.505375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:40.505649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:40.505937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] PATH_INFO: `/v3/users/999c5e5ed9fd4c2082ef4afbaa6af768` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:40.507100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=999c5e5ed9fd4c2082ef4afbaa6af768)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:40.509212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:40.514997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:40.732182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999c5e5ed9fd4c2082ef4afbaa6af768'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.815692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999c5e5ed9fd4c2082ef4afbaa6af768'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.818455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999c5e5ed9fd4c2082ef4afbaa6af768'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.829193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999c5e5ed9fd4c2082ef4afbaa6af768'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.852465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f44502a3-380e-47b6-a49a-073a8e8a7e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999c5e5ed9fd4c2082ef4afbaa6af768'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:40.862483 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1033/2075] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:45:40 2022] DELETE /identity/v3/users/999c5e5ed9fd4c2082ef4afbaa6af768 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:41.365630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:41.367043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:41.402848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24a633-d22d-4769-b16c-80499493d792 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:41.492078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:41.500227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:41.500919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:41.501348 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:41.501632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] PATH_INFO: `/v3/projects/08b331282e5c4282921c747a05f3dd18` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:41.502977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b331282e5c4282921c747a05f3dd18)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:41.505530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:41.697142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b331282e5c4282921c747a05f3dd18', 'request_id': 'req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:41.714732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b331282e5c4282921c747a05f3dd18', 'request_id': 'req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:41.832029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b331282e5c4282921c747a05f3dd18', 'request_id': 'req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:41.850558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] The token cache is being invalidate because project 08b331282e5c4282921c747a05f3dd18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:45:41.850756 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:41.851521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5463fb78-a56f-4257-8d18-c4e3bb42a6ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:45:41.853920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1043/2076] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:45:41 2022] DELETE /identity/v3/projects/08b331282e5c4282921c747a05f3dd18 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:41.858582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:41.859101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:41.859101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:41.890319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:41.892231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:41.997422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1034/2077] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:45:42.006690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.006979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.007260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.040632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:42.041909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:42.052224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1044/2078] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:42.056764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.068304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e745d3-ff4c-4b7a-a645-3e37be37af40 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.075594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.076832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.076832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.076832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.077464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:45:42.077823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.079591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.094757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1035/2079] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:45:42.100223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.111338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bda65a-68fe-41b9-a360-d4ef310b6f0b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.127574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.128222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.128789 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.129497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.130523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.135758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.250899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1045/2080] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:42 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:42.257021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.272434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0dbc1-6eb0-48fb-9c10-3dcfc1c9c7fe admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.358885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.359716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.360170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.360636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.361548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.365013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.483318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1036/2081] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:42 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:45:42.487241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.503193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8336e34-b816-4e85-a711-512f4d5f5448 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.525461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.526911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.527526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.527526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.528447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.530185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.546692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1046/2082] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:42.559255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.569332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:42.576382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c28db-e4d1-4eec-86dd-b1a7ac7a4594 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.583633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5959b96-13b4-47d0-b537-69833bb34f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.584260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5959b96-13b4-47d0-b537-69833bb34f36 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.585382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5959b96-13b4-47d0-b537-69833bb34f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.585382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5959b96-13b4-47d0-b537-69833bb34f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.585711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5959b96-13b4-47d0-b537-69833bb34f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.587493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5959b96-13b4-47d0-b537-69833bb34f36 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.619235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1037/2083] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:42.625145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.640457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086df07f-fac0-4ad0-a7fb-062a298c981e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:42.651335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:42.652679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.653156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.653606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] PATH_INFO: `/v3/projects/7d1070c0033348859723b17e6a97974e/users/bb0b43d2231a45f4a6b369b475e50fcc/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.683016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1070c0033348859723b17e6a97974e, user_id=bb0b43d2231a45f4a6b369b475e50fcc, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:42.686303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:42.787994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1047/2084] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:42 2022] PUT /identity/v3/projects/7d1070c0033348859723b17e6a97974e/users/bb0b43d2231a45f4a6b369b475e50fcc/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:42.793716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff7d1-0ef5-40cc-9772-bfe1b57fd908 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:42.794202 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff7d1-0ef5-40cc-9772-bfe1b57fd908 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:42.796031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ff7d1-0ef5-40cc-9772-bfe1b57fd908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:42.838690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ff7ff7d1-0ef5-40cc-9772-bfe1b57fd908 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:42.840225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ff7ff7d1-0ef5-40cc-9772-bfe1b57fd908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:42.963729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1038/2085] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:42.981775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:42.985604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:43.149383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393ffb9-d185-452f-9a9f-c145aff9d0fe admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:43.161350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:43.162379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:43.174405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:43.175252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:43.194557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:43.196938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:43.227924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1048/2086] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:43.382988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96b92-f979-4be3-b58f-0d80d7c97f7e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:43.383281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96b92-f979-4be3-b58f-0d80d7c97f7e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:43.383556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96b92-f979-4be3-b58f-0d80d7c97f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:43.451178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e4a96b92-f979-4be3-b58f-0d80d7c97f7e None None] MFA Rules not processed for user `fc27435df46542a3ac37e743b6399387`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:43.452697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e4a96b92-f979-4be3-b58f-0d80d7c97f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:43.599318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1039/2087] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:43.616241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:43.619533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:43.691419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:43.758740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e77ee5-3788-4a2c-a5db-3dec43b4226d service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:43.764741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:43.765603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:43.766059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:43.766475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:43.778106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:43.778689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:43.800718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1049/2088] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:44.912059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:44.912666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:44.912666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:44.947642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] MFA Rules not processed for user `bb0b43d2231a45f4a6b369b475e50fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:44.949162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:45.058208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1040/2089] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:45.063006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:45.101140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1bedf-6c4f-4339-8cd1-029da95d0e21 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:45.109575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:45.110991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:45.111495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:45.111995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:45.114251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:45.119880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:45.262678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1050/2090] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:45:45 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:45.264045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:45.281101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401de38-44ab-4e2e-858b-a4291a1a8a12 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:45.294463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:45.358612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:45.359293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:45.359573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:45.360120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:45.360530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:45.362549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:45.457112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1041/2091] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:45:45 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:45:45.464256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:45.491930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5596cb05-b1be-4e9c-9f78-130cd8258ee4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:45.497588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:45.498324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:45.498597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:45.498867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:45.499882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:45.502106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:45.520488 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1051/2092] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:45:45.526938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:45.543969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6455b11-91c4-479a-8e48-a5bd67f9fad5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:45.572705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:45.573324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:45.573602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:45.573871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] PATH_INFO: `/v3/projects/d8c2de3636064a738dcb76af19ebaa99/users/7f73f9fbb75a487eb8d85a8bd64a450c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:45.629227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c2de3636064a738dcb76af19ebaa99, user_id=7f73f9fbb75a487eb8d85a8bd64a450c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:45.636315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:45.712783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1042/2093] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:45 2022] PUT /identity/v3/projects/d8c2de3636064a738dcb76af19ebaa99/users/7f73f9fbb75a487eb8d85a8bd64a450c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:45.716931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:45.728867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc09186-31ff-456c-a020-13db37b7a635 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:45.818452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd5b66-987d-48be-b612-9629e6ed23ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:45.820713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd5b66-987d-48be-b612-9629e6ed23ef admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:45.821300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd5b66-987d-48be-b612-9629e6ed23ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:45.821933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd5b66-987d-48be-b612-9629e6ed23ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:45.823588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd5b66-987d-48be-b612-9629e6ed23ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:45.827601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd5b66-987d-48be-b612-9629e6ed23ef admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:45.852451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1052/2094] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:45:45.856714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:45.870217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d69207-64a7-4801-8b03-08dd6eb3fe03 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:45.906894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc9e6c-6679-4fa9-a132-bf3cfb57650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:45.911070 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9e6c-6679-4fa9-a132-bf3cfb57650b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:45.911356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9e6c-6679-4fa9-a132-bf3cfb57650b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:45.911638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc9e6c-6679-4fa9-a132-bf3cfb57650b admin admin] PATH_INFO: `/v3/projects/d8c2de3636064a738dcb76af19ebaa99/users/7f73f9fbb75a487eb8d85a8bd64a450c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:45.917527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc9e6c-6679-4fa9-a132-bf3cfb57650b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c2de3636064a738dcb76af19ebaa99, user_id=7f73f9fbb75a487eb8d85a8bd64a450c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:45.921916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc9e6c-6679-4fa9-a132-bf3cfb57650b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:45.951081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1043/2095] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:45:45 2022] PUT /identity/v3/projects/d8c2de3636064a738dcb76af19ebaa99/users/7f73f9fbb75a487eb8d85a8bd64a450c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:45:46.982255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:46.983059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:46.983059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:47.016801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] MFA Rules not processed for user `fc27435df46542a3ac37e743b6399387`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:47.020179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:47.126665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1053/2096] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:47.626452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:47.630794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:47.630794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:47.710495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] MFA Rules not processed for user `7f73f9fbb75a487eb8d85a8bd64a450c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:45:47.712087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:47.844765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1044/2097] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:45:47.860662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:47.860662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:45:47.920124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:45:47.994140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c227-b176-47ed-ace2-e36f23f42fd1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:47.998921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:47.999566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:48.000469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:48.001150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:48.119705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:48.122739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:48.137155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1054/2098] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:45:50.171463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:45:50.191907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5681b9-1ba7-40ba-8e51-f95ffb7addb2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:45:50.196239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fb248-cdf7-414d-8102-4cf8ae5a354b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:45:50.196937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fb248-cdf7-414d-8102-4cf8ae5a354b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:45:50.197215 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fb248-cdf7-414d-8102-4cf8ae5a354b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:45:50.197488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fb248-cdf7-414d-8102-4cf8ae5a354b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:45:50.210201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fb248-cdf7-414d-8102-4cf8ae5a354b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:45:50.211699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fb248-cdf7-414d-8102-4cf8ae5a354b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:45:50.225592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1045/2099] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:45:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:45:58.131749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=39 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:00.221595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:03.812456 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:03.815151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:03.902396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8be0d-70a6-4470-9137-7423d47437b7 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:03.908283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:03.909044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:03.909482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:03.909765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] PATH_INFO: `/v3/users/4abc64377cf54df1b2da448fc3bae463` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:03.911408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4abc64377cf54df1b2da448fc3bae463)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:03.913757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:04.466555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4abc64377cf54df1b2da448fc3bae463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:04.468319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4abc64377cf54df1b2da448fc3bae463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:04.493830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4abc64377cf54df1b2da448fc3bae463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:04.662617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4abc64377cf54df1b2da448fc3bae463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:04.663388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4abc64377cf54df1b2da448fc3bae463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:04.683445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1055/2100] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:46:03 2022] DELETE /identity/v3/users/4abc64377cf54df1b2da448fc3bae463 => generated 0 bytes in 872 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:04.687960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:04.688412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:04.688504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:04.708138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:04.709376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:04.774664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1046/2101] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:04.789563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:04.791030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:04.872746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354203e-513e-4c19-a5ff-51c2409b002e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:04.876967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0ceb4-a185-4905-a9b8-62456775530f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:04.877644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ceb4-a185-4905-a9b8-62456775530f service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:04.877985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ceb4-a185-4905-a9b8-62456775530f service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:04.878352 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ceb4-a185-4905-a9b8-62456775530f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:04.893904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ceb4-a185-4905-a9b8-62456775530f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:04.895376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ceb4-a185-4905-a9b8-62456775530f service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:04.905950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1056/2102] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:05.446594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:05.448662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:05.471314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310375a-4738-4309-8327-4d97f4a0c43d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:05.476717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:05.477726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:05.477997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:05.478314 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] PATH_INFO: `/v3/projects/b1017dfe87f64ef69cad719ccabda993` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:05.479607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1017dfe87f64ef69cad719ccabda993)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:05.481619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:05.597586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1017dfe87f64ef69cad719ccabda993', 'request_id': 'req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:05.668310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1017dfe87f64ef69cad719ccabda993', 'request_id': 'req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:05.676959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1017dfe87f64ef69cad719ccabda993', 'request_id': 'req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:05.694457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] The token cache is being invalidate because project b1017dfe87f64ef69cad719ccabda993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:46:05.694457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:05.694457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e61a45c6-3e82-4bcf-984d-dd4ba4d572c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:05.694457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1047/2103] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:46:05 2022] DELETE /identity/v3/projects/b1017dfe87f64ef69cad719ccabda993 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:05.697310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:05.697602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:05.697868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:05.719942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:05.724965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:05.740366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:05.786569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1057/2104] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:05.794777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:05.795134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:05.795459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:05.819001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:05.820963 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:05.830253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1048/2105] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:05.834534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:05.844309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa63052-ce0b-409d-b16b-2642d0785e6a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:05.849847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:05.850461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:05.850741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:05.851045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:05.851692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:46:05.851992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:05.853640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:05.867068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1058/2106] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:46:05.880325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:05.890261 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec64a3d-933a-45b9-8446-e20915c36520 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:05.903614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:05.903614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:05.903614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:05.903614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:05.903959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:05.904076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.037628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1049/2107] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:05 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:06.042055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:06.053671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7dea4-2e2e-4abf-bda3-a77918519dd7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:06.117775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:06.118459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.118735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.119039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.119740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:06.121693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.252690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1059/2108] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:06 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:46:06.257235 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:06.268085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b3c92-fbc8-49a8-8a87-8f791adce882 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:06.273221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:06.273836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.274117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.274388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.275042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:06.276729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.287649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1050/2109] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:06.292255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:06.302948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e77734-4c6c-4a29-9fc1-45281b6fe650 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:06.307665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233e369-5113-430a-a293-07ee63420f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:06.308270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e369-5113-430a-a293-07ee63420f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.308587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e369-5113-430a-a293-07ee63420f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.308857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233e369-5113-430a-a293-07ee63420f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.309500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233e369-5113-430a-a293-07ee63420f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:06.311325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233e369-5113-430a-a293-07ee63420f10 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.326729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1060/2110] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:46:06.327959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:06.338358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f3dbc-fd1b-455c-aeb6-ff9b6c1d9d00 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:06.342885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:06.343522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.343798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.344067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] PATH_INFO: `/v3/projects/05a8a1a0e0334559ab3fc9a63685fa69/users/5f10bf98bc784be7bf2f8fde1ddbef44/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.359662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a8a1a0e0334559ab3fc9a63685fa69, user_id=5f10bf98bc784be7bf2f8fde1ddbef44, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:06.362907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.484023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1051/2111] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:06 2022] PUT /identity/v3/projects/05a8a1a0e0334559ab3fc9a63685fa69/users/5f10bf98bc784be7bf2f8fde1ddbef44/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:06.486299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.494168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.494168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.506993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:06.508227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:06.575525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1061/2112] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:06.582396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:06.584455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:06.633022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:06.670907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc756fd-5484-4d9a-af6f-ce30be67c40e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:06.675688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1eac98-b6ba-4403-97b7-b39e40d0d0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:06.676296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eac98-b6ba-4403-97b7-b39e40d0d0ec service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.676601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eac98-b6ba-4403-97b7-b39e40d0d0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.676865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1eac98-b6ba-4403-97b7-b39e40d0d0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.686923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eac98-b6ba-4403-97b7-b39e40d0d0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:06.688380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1eac98-b6ba-4403-97b7-b39e40d0d0ec service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.699037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1052/2113] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:06.870335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:06.872011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:06.884662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e630e-cdbb-4a90-af7e-5297707be328 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:06.889924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:06.890771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:06.894064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:06.894064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] PATH_INFO: `/v3/users/6f1c266a87c147d598bf327b56556852` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:06.894064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1c266a87c147d598bf327b56556852)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:06.894990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:06.919218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:07.157220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1c266a87c147d598bf327b56556852'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:07.164081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1c266a87c147d598bf327b56556852'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:07.166468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1c266a87c147d598bf327b56556852'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:07.239213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1c266a87c147d598bf327b56556852'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:07.239947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1c266a87c147d598bf327b56556852'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:07.261455 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1062/2114] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:46:06 2022] DELETE /identity/v3/users/6f1c266a87c147d598bf327b56556852 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:07.265811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:07.266093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:07.266775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:07.287521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:07.288937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:07.361510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1053/2115] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:07.372021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:07.373498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:07.432258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8787626-7f03-4ca6-978a-04736cd8a849 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:07.436027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979eb83-676e-46f1-9f02-e5f7d89bd669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:07.436665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979eb83-676e-46f1-9f02-e5f7d89bd669 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:07.436948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979eb83-676e-46f1-9f02-e5f7d89bd669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:07.437216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979eb83-676e-46f1-9f02-e5f7d89bd669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:07.446638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979eb83-676e-46f1-9f02-e5f7d89bd669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:07.448542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979eb83-676e-46f1-9f02-e5f7d89bd669 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:07.458477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1063/2116] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:46:08.153264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:08.155933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:08.182493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f4975-4e01-4153-85c1-08958678b4f4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:08.213682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:08.214841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.215668 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.216211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] PATH_INFO: `/v3/projects/db565314dd8c4d6cbe65c13dbea4cdc4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.221338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db565314dd8c4d6cbe65c13dbea4cdc4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:08.223602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:08.362689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db565314dd8c4d6cbe65c13dbea4cdc4', 'request_id': 'req-407c074b-da40-483f-b2ff-e88f0ed86739'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:08.496747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db565314dd8c4d6cbe65c13dbea4cdc4', 'request_id': 'req-407c074b-da40-483f-b2ff-e88f0ed86739'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:08.506384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db565314dd8c4d6cbe65c13dbea4cdc4', 'request_id': 'req-407c074b-da40-483f-b2ff-e88f0ed86739'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:08.523976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] The token cache is being invalidate because project db565314dd8c4d6cbe65c13dbea4cdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:46:08.524486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:08.525807 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-407c074b-da40-483f-b2ff-e88f0ed86739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:08.531404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1054/2117] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:46:08 2022] DELETE /identity/v3/projects/db565314dd8c4d6cbe65c13dbea4cdc4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:08.537368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.537901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.542450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.567913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:08.571709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:08.669409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1064/2118] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:08.678822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.679359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.683026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.705254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:08.706860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:08.716405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1055/2119] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:08.723360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:08.731378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eac10c-4ccf-44fe-9b05-48cd268117b3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:08.737401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:08.738017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.738296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.738568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.739228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:46:08.739533 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:08.744588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:08.759704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1065/2120] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:46:08.764687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:08.775921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a8066-f90f-4ae7-95e2-a026cd862440 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:08.780800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:08.781414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:08.781691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:08.781958 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:08.783871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:08.785859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:08.942986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1056/2121] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:08 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:08.944555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:08.959205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1938326-ebda-48d0-8922-80a12a6000ba admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:09.032284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:09.062926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:09.064685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.065413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.065996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.067384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:09.070442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:09.215038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1066/2122] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:08 2022] POST /identity/v3/users => generated 288 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:09.221637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:09.251131 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd8168-2fcf-4f12-82cf-b789515819c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:09.281689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:09.287628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.287628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.287628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.287628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:09.287628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:09.335609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1057/2123] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:09.339476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:09.354042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629bc16-4b37-485f-b3e2-fb7f5c5f45a6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:09.359394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdbd6e-6675-4aa7-8707-2d9c6dd3570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:09.361521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdbd6e-6675-4aa7-8707-2d9c6dd3570e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.361908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdbd6e-6675-4aa7-8707-2d9c6dd3570e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.362296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdbd6e-6675-4aa7-8707-2d9c6dd3570e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.363327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdbd6e-6675-4aa7-8707-2d9c6dd3570e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:09.365228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdbd6e-6675-4aa7-8707-2d9c6dd3570e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:09.389438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1067/2124] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:09.393686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:09.408720 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f5ba-469b-4f19-8f95-5a8aa5bd97e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:09.425282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:09.426096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.426480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.426855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] PATH_INFO: `/v3/projects/bbd583b569f04997b2ee33b74110ea25/users/ec16281c5cfd4c2a90b12c097c0a3e0a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.432129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:09.448772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd583b569f04997b2ee33b74110ea25, user_id=ec16281c5cfd4c2a90b12c097c0a3e0a, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:09.452921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:09.545148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1058/2125] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:09 2022] PUT /identity/v3/projects/bbd583b569f04997b2ee33b74110ea25/users/ec16281c5cfd4c2a90b12c097c0a3e0a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:09.550172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5e0f-8b41-4281-87de-ced90c7dbfbb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.551312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5e0f-8b41-4281-87de-ced90c7dbfbb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.551674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5e0f-8b41-4281-87de-ced90c7dbfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.633054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-45ab5e0f-8b41-4281-87de-ced90c7dbfbb None None] MFA Rules not processed for user `ec16281c5cfd4c2a90b12c097c0a3e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:09.635108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-45ab5e0f-8b41-4281-87de-ced90c7dbfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:09.750411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1068/2126] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:09.768424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:09.770337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:09.853560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3fbbe-45ed-4bac-b1a6-feed834221b4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:09.859794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0225a1-4ce3-4aa4-9075-d243d2168b25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:09.861049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0225a1-4ce3-4aa4-9075-d243d2168b25 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:09.861415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0225a1-4ce3-4aa4-9075-d243d2168b25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:09.861774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0225a1-4ce3-4aa4-9075-d243d2168b25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:09.872679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0225a1-4ce3-4aa4-9075-d243d2168b25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:09.877778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0225a1-4ce3-4aa4-9075-d243d2168b25 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:09.905015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1059/2127] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:11.941620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:11.942127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:11.942219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:11.996135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] MFA Rules not processed for user `ec16281c5cfd4c2a90b12c097c0a3e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:11.996135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:12.008208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1069/2128] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:14.224052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:14.226043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:14.226043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:14.271689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] MFA Rules not processed for user `5f10bf98bc784be7bf2f8fde1ddbef44`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:14.283623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:14.406494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1060/2129] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:14.411460 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:14.413542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:14.493865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2019a5-cd00-4f65-b8fa-84060f33da01 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:14.515442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:14.515442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:14.515442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:14.515442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:14.516880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:14.517194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:14.629983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1070/2130] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:14 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:14.634576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:14.647079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862cbc84-13f3-45c3-a208-1dbd15073343 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:14.726736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:14.727268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:14.727847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:14.727847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:14.728525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:14.730497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:14.867244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1061/2131] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:14 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:14.875275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:14.883293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eabf3-9586-4316-919b-2f3a2677e677 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:14.888796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:14.889628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:14.890003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:14.890375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:14.891961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:14.894429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:14.907213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1071/2132] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:14.911464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:14.918330 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:14.922100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb311c-8265-4e57-a19e-e452adf8f0e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:14.927140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:14.927864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:14.928211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:14.928642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] PATH_INFO: `/v3/projects/fa06923c74854d6c82a342edd8a670af/users/0f449d61d6b141feb0cf65f4f89f35d1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:14.948902 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa06923c74854d6c82a342edd8a670af, user_id=0f449d61d6b141feb0cf65f4f89f35d1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:14.951852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:15.141673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1062/2133] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:14 2022] PUT /identity/v3/projects/fa06923c74854d6c82a342edd8a670af/users/0f449d61d6b141feb0cf65f4f89f35d1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:15.146028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:15.170088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b0e64-f594-4b62-92f3-00ead8da76f8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:15.176910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:15.275719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b688f-a7dc-4cf5-ae25-83896098db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:15.276368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b688f-a7dc-4cf5-ae25-83896098db0d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.276697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b688f-a7dc-4cf5-ae25-83896098db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.277041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b688f-a7dc-4cf5-ae25-83896098db0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.280602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b688f-a7dc-4cf5-ae25-83896098db0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:15.283597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b688f-a7dc-4cf5-ae25-83896098db0d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:15.297739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1072/2134] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:15.299029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:15.310848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3f948-fe74-4035-90e6-cfe5515f8daf admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:15.316713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c104ba-a44f-47ca-8f9b-7c45b1313af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:15.317690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c104ba-a44f-47ca-8f9b-7c45b1313af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.317973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c104ba-a44f-47ca-8f9b-7c45b1313af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.318446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c104ba-a44f-47ca-8f9b-7c45b1313af3 admin admin] PATH_INFO: `/v3/projects/fa06923c74854d6c82a342edd8a670af/users/0f449d61d6b141feb0cf65f4f89f35d1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.320633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c104ba-a44f-47ca-8f9b-7c45b1313af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa06923c74854d6c82a342edd8a670af, user_id=0f449d61d6b141feb0cf65f4f89f35d1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:15.324277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c104ba-a44f-47ca-8f9b-7c45b1313af3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:15.338650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1063/2135] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:15 2022] PUT /identity/v3/projects/fa06923c74854d6c82a342edd8a670af/users/0f449d61d6b141feb0cf65f4f89f35d1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:15.428521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e61d55-c45a-4ffc-b732-2689191dc495 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.429008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e61d55-c45a-4ffc-b732-2689191dc495 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.429008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e61d55-c45a-4ffc-b732-2689191dc495 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.430322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1073/2136] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:46:15.436136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.436445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.436727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.458515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] MFA Rules not processed for user `0234ca49360c4a7cbbaec848f27b89ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:15.459961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:15.556513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1064/2137] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:46:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:15.569541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e61d55-c45a-4ffc-b732-2689191dc495 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:15.572648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-95e61d55-c45a-4ffc-b732-2689191dc495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:15.592109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e61d55-c45a-4ffc-b732-2689191dc495 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:15.603205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:15.604013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:15.604299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:15.604651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:15.614897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:15.616573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:15.664299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1074/2138] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:16.699620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:16.701122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:16.798312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b894b05-88e6-4a6b-974f-51db7b1c808f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:16.803217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:16.804139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:16.804533 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:16.804870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] PATH_INFO: `/v3/users/bb0b43d2231a45f4a6b369b475e50fcc` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:16.806107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0b43d2231a45f4a6b369b475e50fcc)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:16.808411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:17.081047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0b43d2231a45f4a6b369b475e50fcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:17.164491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0b43d2231a45f4a6b369b475e50fcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:17.165985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0b43d2231a45f4a6b369b475e50fcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:17.175412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0b43d2231a45f4a6b369b475e50fcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:17.177523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:17.188782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0b43d2231a45f4a6b369b475e50fcc'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:17.203128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1065/2139] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:46:16 2022] DELETE /identity/v3/users/bb0b43d2231a45f4a6b369b475e50fcc => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:17.606592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:17.627785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558546a3-aafd-467f-9d27-3341e10d0099 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:17.762421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:17.764257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:17.770583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:17.770583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] PATH_INFO: `/v3/projects/7d1070c0033348859723b17e6a97974e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:17.770583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1070c0033348859723b17e6a97974e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:17.773921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:18.000003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1070c0033348859723b17e6a97974e', 'request_id': 'req-06f07893-d4cd-4958-8435-06a3237027dd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.016482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1070c0033348859723b17e6a97974e', 'request_id': 'req-06f07893-d4cd-4958-8435-06a3237027dd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.187403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1070c0033348859723b17e6a97974e', 'request_id': 'req-06f07893-d4cd-4958-8435-06a3237027dd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.213325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] The token cache is being invalidate because project 7d1070c0033348859723b17e6a97974e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:46:18.213554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.215453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.218531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1075/2140] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:46:17 2022] DELETE /identity/v3/projects/7d1070c0033348859723b17e6a97974e => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:18.229568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:18.231841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:18.257049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67e37-e70b-4a84-8e90-5009aa506e9f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:18.404644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:18.408736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:18.412090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:18.412557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] PATH_INFO: `/v3/users/7f73f9fbb75a487eb8d85a8bd64a450c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:18.416013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f73f9fbb75a487eb8d85a8bd64a450c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:18.418212 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:18.650604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f73f9fbb75a487eb8d85a8bd64a450c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.749609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f73f9fbb75a487eb8d85a8bd64a450c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.751453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f73f9fbb75a487eb8d85a8bd64a450c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.776210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f73f9fbb75a487eb8d85a8bd64a450c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.850750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-83c0acc3-c9bc-41ec-ae68-3e85a9649679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f73f9fbb75a487eb8d85a8bd64a450c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:18.883713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1066/2141] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:46:18 2022] DELETE /identity/v3/users/7f73f9fbb75a487eb8d85a8bd64a450c => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:19.705835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:19.708175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:19.728391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07893-d4cd-4958-8435-06a3237027dd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:19.753055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:19.857765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:19.858421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:19.858702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:19.861464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] PATH_INFO: `/v3/projects/d8c2de3636064a738dcb76af19ebaa99` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:19.863256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c2de3636064a738dcb76af19ebaa99)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:19.865249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:20.078799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c2de3636064a738dcb76af19ebaa99', 'request_id': 'req-0abb0500-4df7-4c5a-a249-c91dd3c1276e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:20.090906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c2de3636064a738dcb76af19ebaa99', 'request_id': 'req-0abb0500-4df7-4c5a-a249-c91dd3c1276e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:20.210558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c2de3636064a738dcb76af19ebaa99', 'request_id': 'req-0abb0500-4df7-4c5a-a249-c91dd3c1276e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:20.231409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] The token cache is being invalidate because project d8c2de3636064a738dcb76af19ebaa99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:46:20.232038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:20.232916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0abb0500-4df7-4c5a-a249-c91dd3c1276e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:20.238921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1076/2142] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:46:19 2022] DELETE /identity/v3/projects/d8c2de3636064a738dcb76af19ebaa99 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:20.243099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:20.243099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:20.243099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:20.277809 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:20.279098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:20.357088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1067/2143] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:20.365184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:20.365724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:20.366143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:20.393372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:20.395508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:20.408744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1077/2144] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:20.413292 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:20.420107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:20.430199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5fdb1-5e94-4cd3-9a5f-323d150de7c9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:20.439611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:20.440710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:20.442197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:20.444163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:20.447488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:46:20.448260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:20.450630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:20.472787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1068/2145] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:46:20.477484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:20.497297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26105f3c-236b-4026-a87e-27caf6fdb37c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:20.510172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:20.510814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:20.511100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:20.511374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:20.512019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:20.514011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:20.668135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1078/2146] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:20 2022] POST /identity/v3/projects => generated 363 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:20.669300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:20.683920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190ed46-6b3c-4749-bf80-e4730eec60d5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:20.877406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:20.878587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:20.878815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:20.879091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:20.880770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:20.882946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:21.174322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:21.174888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1069/2147] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:20 2022] POST /identity/v3/users => generated 282 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:21.192284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7be410-d032-4a71-942d-32a91038d35d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:21.198353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:21.199167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:21.199439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:21.199871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:21.200876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:21.203504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:21.217803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1079/2148] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:46:21.222601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:21.240456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e442f-58a2-4f9e-890d-542adc1e87de admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:21.251916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdf890-01c6-4651-86ca-8cdf60509f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:21.253671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdf890-01c6-4651-86ca-8cdf60509f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:21.254098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdf890-01c6-4651-86ca-8cdf60509f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:21.254853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdf890-01c6-4651-86ca-8cdf60509f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:21.255803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdf890-01c6-4651-86ca-8cdf60509f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:21.259890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdf890-01c6-4651-86ca-8cdf60509f46 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:21.276209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1070/2149] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:21.280206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:21.293374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e7318-0d18-453d-9416-dc04622e1b23 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:21.303686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8047cc-fed1-440f-a7ea-fe2c040f5f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:21.305885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8047cc-fed1-440f-a7ea-fe2c040f5f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:21.305885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8047cc-fed1-440f-a7ea-fe2c040f5f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:21.305885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8047cc-fed1-440f-a7ea-fe2c040f5f82 admin admin] PATH_INFO: `/v3/projects/9aa80633bc9c4d98a53a31bd433c45d3/users/4ac38e3eccb84a6a8a6800ef178a4dc8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:21.321099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8047cc-fed1-440f-a7ea-fe2c040f5f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa80633bc9c4d98a53a31bd433c45d3, user_id=4ac38e3eccb84a6a8a6800ef178a4dc8, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:21.323927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8047cc-fed1-440f-a7ea-fe2c040f5f82 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:21.479562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1080/2150] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:21 2022] PUT /identity/v3/projects/9aa80633bc9c4d98a53a31bd433c45d3/users/4ac38e3eccb84a6a8a6800ef178a4dc8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:46:21.480937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:21.481226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:21.481501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:21.535554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] MFA Rules not processed for user `4ac38e3eccb84a6a8a6800ef178a4dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:21.537020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:21.650596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ad53-e22b-4d2c-9028-cc677c8074cd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:21.651863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ad53-e22b-4d2c-9028-cc677c8074cd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:21.652969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ad53-e22b-4d2c-9028-cc677c8074cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:21.665974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1071/2151] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:46:21.674327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:21.675813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:21.695896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-44f1ad53-e22b-4d2c-9028-cc677c8074cd None None] MFA Rules not processed for user `7be50b041b5947d5909ed6ce516ec8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:21.695896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-44f1ad53-e22b-4d2c-9028-cc677c8074cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:21.756064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f35f86-e88c-480c-8c05-eb47f5e4bb72 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:21.761336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fec79-83ca-45c3-9f37-4039f8781687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:21.761957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fec79-83ca-45c3-9f37-4039f8781687 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:21.762278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fec79-83ca-45c3-9f37-4039f8781687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:21.763062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fec79-83ca-45c3-9f37-4039f8781687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:21.774342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fec79-83ca-45c3-9f37-4039f8781687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:21.776604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fec79-83ca-45c3-9f37-4039f8781687 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:21.781379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:21.792140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1072/2152] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:46:21.808813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1081/2153] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:46:22.714748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:25.024718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:25.025283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:25.026960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:25.057739 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] MFA Rules not processed for user `1d9f7755863d45bfb04a1498bd62e367`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:25.059249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:25.169689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1073/2154] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:25.175912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b3e91-d011-464b-ba5c-b63fdb60d76e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:25.176462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b3e91-d011-464b-ba5c-b63fdb60d76e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:25.182509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b3e91-d011-464b-ba5c-b63fdb60d76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:25.221278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8c7b3e91-d011-464b-ba5c-b63fdb60d76e None None] MFA Rules not processed for user `0f449d61d6b141feb0cf65f4f89f35d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:25.223672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8c7b3e91-d011-464b-ba5c-b63fdb60d76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:25.348758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1082/2155] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:25.363174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:25.366307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:25.497652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc39d13-d8fe-4b21-aa64-30981393dfc4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:25.497652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:25.497652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:25.497652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:25.498240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:25.499315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:25.595071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:25.597469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc5952-c359-42b2-b885-60f107ddc15b service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:25.608220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1074/2156] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:46:29.179287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:29.179857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:29.179857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:29.211024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] MFA Rules not processed for user `5f10bf98bc784be7bf2f8fde1ddbef44`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:29.215559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:29.220830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1083/2157] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:35.603507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:39.203443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=41 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:42.403845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9aba9-7d0b-4d29-8f6b-6ef959973dfd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:42.404319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9aba9-7d0b-4d29-8f6b-6ef959973dfd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:42.407045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9aba9-7d0b-4d29-8f6b-6ef959973dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:42.425424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3ae9aba9-7d0b-4d29-8f6b-6ef959973dfd None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:42.427082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3ae9aba9-7d0b-4d29-8f6b-6ef959973dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:42.485692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1075/2158] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:46:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:42.528612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:42.530191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:42.607144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af171c6-5c68-446d-81a8-90a92f852f2f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:42.613777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:42.614452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:42.614751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:42.615212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:42.630958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:42.632599 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:42.641913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1084/2159] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:46:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:46:43.666069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f05e7-c48f-4628-a94c-ea3fa447ccb8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:43.666573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f05e7-c48f-4628-a94c-ea3fa447ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:43.666670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f05e7-c48f-4628-a94c-ea3fa447ccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:43.685981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-314f05e7-c48f-4628-a94c-ea3fa447ccb8 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:43.687207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-314f05e7-c48f-4628-a94c-ea3fa447ccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:43.694591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1076/2160] 10.222.0.17 () {58 vars in 967 bytes} [Wed Aug 24 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:43.710051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:43.722586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bca8f-2077-4308-8d3f-43b6f1711d4d service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:43.727728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:43.728592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:43.731815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:43.732094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:43.743138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:43.744779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:43.758297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1085/2161] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:46:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:46:44.646003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c19324-d8f1-4d28-9f3e-b8eab9ec17ee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:44.646285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c19324-d8f1-4d28-9f3e-b8eab9ec17ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:44.646763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c19324-d8f1-4d28-9f3e-b8eab9ec17ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:44.669090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-20c19324-d8f1-4d28-9f3e-b8eab9ec17ee None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:44.671437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-20c19324-d8f1-4d28-9f3e-b8eab9ec17ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:44.684509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1077/2162] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:46:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:44.696512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:44.708417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110f6b2-bbfb-4aac-a04d-c9ff9e1623ef service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:44.714837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:44.715933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:44.716225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:44.716563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:44.727079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:44.728705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:44.737906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1086/2163] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:46:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:46:45.511472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:45.511757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:45.512030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:45.532808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:45.534265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:45.541974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1078/2164] 10.222.0.17 () {58 vars in 967 bytes} [Wed Aug 24 16:46:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:45.561185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:45.572476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d8d3c-e79b-4626-840a-82bb6dc04b16 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:45.578829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad88ea-341f-4d25-9f13-77c34a5b4694 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:45.579535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad88ea-341f-4d25-9f13-77c34a5b4694 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:45.579882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad88ea-341f-4d25-9f13-77c34a5b4694 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:45.580223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad88ea-341f-4d25-9f13-77c34a5b4694 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:45.589366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad88ea-341f-4d25-9f13-77c34a5b4694 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:45.591269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad88ea-341f-4d25-9f13-77c34a5b4694 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:45.601110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1087/2165] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:46:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:46:48.566855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:48.572521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:48.648229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed11421-836a-4869-8e65-999f803b3a4c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:48.652941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:48.653613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:48.653882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:48.654380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] PATH_INFO: `/v3/users/ec16281c5cfd4c2a90b12c097c0a3e0a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:48.656731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec16281c5cfd4c2a90b12c097c0a3e0a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:48.659115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:48.892163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec16281c5cfd4c2a90b12c097c0a3e0a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:48.987429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec16281c5cfd4c2a90b12c097c0a3e0a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:48.987999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec16281c5cfd4c2a90b12c097c0a3e0a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:48.996132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec16281c5cfd4c2a90b12c097c0a3e0a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.011692 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec16281c5cfd4c2a90b12c097c0a3e0a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.020332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1079/2166] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:46:48 2022] DELETE /identity/v3/users/ec16281c5cfd4c2a90b12c097c0a3e0a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:49.024955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:49.025517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:49.025517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:49.048466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:49.048466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:49.135386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1088/2167] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:46:49.148944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:49.150492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:49.157661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:49.214351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cf53-ecdd-488a-8aee-eaabcbb48dae admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:49.218027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be4b23-8b0f-4acb-a3d2-4960dd4be7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:49.219111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be4b23-8b0f-4acb-a3d2-4960dd4be7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:49.219437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be4b23-8b0f-4acb-a3d2-4960dd4be7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:49.219752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be4b23-8b0f-4acb-a3d2-4960dd4be7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:49.232190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be4b23-8b0f-4acb-a3d2-4960dd4be7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:49.233955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be4b23-8b0f-4acb-a3d2-4960dd4be7a9 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:49.244843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1080/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:49.547151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:49.548719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:49.563060 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2df82-393b-4e91-ad44-90612bfa9fc9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:49.568873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:49.569638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:49.570093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:49.570369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] PATH_INFO: `/v3/projects/bbd583b569f04997b2ee33b74110ea25` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:49.571519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd583b569f04997b2ee33b74110ea25)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:49.573764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:49.712263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:49.715278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd583b569f04997b2ee33b74110ea25', 'request_id': 'req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.722393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd583b569f04997b2ee33b74110ea25', 'request_id': 'req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.792606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd583b569f04997b2ee33b74110ea25', 'request_id': 'req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.813114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] The token cache is being invalidate because project bbd583b569f04997b2ee33b74110ea25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:46:49.813402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.814544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a2a65e1-c324-46c9-9ec2-0a28499fe4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:46:49.815690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1089/2169] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:46:49 2022] DELETE /identity/v3/projects/bbd583b569f04997b2ee33b74110ea25 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:49.820571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:49.821046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:49.821135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:49.842016 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:49.843833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:49.927067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1081/2170] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:49.934947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:49.935235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:49.936027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:49.960035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:49.960231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:49.970204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1090/2171] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:49.974251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:49.986600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9254-25d9-4404-b083-174e3bd2ab82 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:49.992601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:49.993218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:49.993494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:49.993766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:49.994430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:46:49.994731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:49.996387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.012505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1082/2172] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:46:50.018246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:50.028218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ff015-f182-40e5-9056-284eed4f5461 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:50.035307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:50.036288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.036670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.037030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.037922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:50.041070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.170104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1091/2173] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:50 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:50.175191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:50.191747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d73d-b41f-4ddf-a1fa-cb56eba07cc6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:50.286030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:50.286326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.286680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.287026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.287742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:50.290136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.427587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1083/2174] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:50 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:46:50.429586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:50.446665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36ad22-1808-446e-bfc0-b327a4f465bb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:50.456500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:50.456500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.456500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.456500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.456881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:50.457434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.474178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1092/2175] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:50.477707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:50.491774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6367b-48b3-4c66-9a2d-dd4e62348023 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:50.500124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e0f9-603f-43e0-b07b-8f7765b4e4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:50.501285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e0f9-603f-43e0-b07b-8f7765b4e4cb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.501285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e0f9-603f-43e0-b07b-8f7765b4e4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.501665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e0f9-603f-43e0-b07b-8f7765b4e4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.502407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e0f9-603f-43e0-b07b-8f7765b4e4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:50.504462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e0f9-603f-43e0-b07b-8f7765b4e4cb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.522404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1084/2176] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:46:50.523718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:50.534717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cad7c-b376-4420-b551-7bec479ae215 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:50.539644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:50.540273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.540597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.540870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] PATH_INFO: `/v3/projects/09e2ab2c313d4448a28a6902b74a0d10/users/680f726c9e374d97aff5ebe924c84865/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.558620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e2ab2c313d4448a28a6902b74a0d10, user_id=680f726c9e374d97aff5ebe924c84865, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:50.562070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.637522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1093/2177] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:50 2022] PUT /identity/v3/projects/09e2ab2c313d4448a28a6902b74a0d10/users/680f726c9e374d97aff5ebe924c84865/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:50.643713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd322df-27e3-48ae-ad4f-b84a5b83f455 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.643998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd322df-27e3-48ae-ad4f-b84a5b83f455 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.644273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd322df-27e3-48ae-ad4f-b84a5b83f455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.667732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dfd322df-27e3-48ae-ad4f-b84a5b83f455 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:50.669060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dfd322df-27e3-48ae-ad4f-b84a5b83f455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:50.678310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:50.741453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1085/2178] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:50.752980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:50.755775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:50.868471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b0f84-acf2-4b34-a700-a49c33db0f4c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:50.873081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:50.874546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:50.874546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:50.874725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:50.886129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:50.887687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:50.901505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1094/2179] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:46:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:46:58.988072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:58.989704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:58.990248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.030970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] MFA Rules not processed for user `680f726c9e374d97aff5ebe924c84865`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:46:59.032974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:46:59.116732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1086/2180] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:46:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:46:59.121396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:59.133857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6975b-827c-4813-afb2-97ac15c66e83 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:59.139244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:59.139864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:59.140141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:59.140497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.141845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:59.143696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:59.240848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1095/2181] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:46:59 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:46:59.245165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:59.257463 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ee8d6-0e33-4648-a2a9-48a70f76ab6e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:59.326553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:59.327170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:59.327451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:59.327738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.329050 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:59.331762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:59.425697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1087/2182] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:46:59 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:46:59.429693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:59.441557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b83e60-d946-4ac0-a491-4787d9813c5e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:59.447335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:59.447517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:59.447795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:59.448069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.449065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:59.450780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:59.466598 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1096/2183] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:46:59.467841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:59.482358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312822b-a76a-4900-b07e-006380f68964 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:59.488708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:59.489359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:59.489628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:59.489901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] PATH_INFO: `/v3/projects/e28414c9956e43dbb9d6c35bddba5dfb/users/d2f14bd46a68406aad1bef377a5337f6/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.508008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28414c9956e43dbb9d6c35bddba5dfb, user_id=d2f14bd46a68406aad1bef377a5337f6, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:59.510759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:59.612961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1088/2184] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:59 2022] PUT /identity/v3/projects/e28414c9956e43dbb9d6c35bddba5dfb/users/d2f14bd46a68406aad1bef377a5337f6/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:46:59.617207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:59.628183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ab009-3044-4170-843b-c35ba991b5e6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:59.675463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:59.714523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95a121-3702-4991-b211-9b9bb2f95a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:59.716041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95a121-3702-4991-b211-9b9bb2f95a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:59.716506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95a121-3702-4991-b211-9b9bb2f95a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:59.717015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95a121-3702-4991-b211-9b9bb2f95a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.717749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95a121-3702-4991-b211-9b9bb2f95a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:59.720019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95a121-3702-4991-b211-9b9bb2f95a1a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:59.734244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1097/2185] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:46:59.736669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:46:59.750845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d183062-1ee4-41e1-8c95-279d4dd9863d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:46:59.758555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:46:59.759240 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:46:59.759517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:46:59.759792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] PATH_INFO: `/v3/projects/e28414c9956e43dbb9d6c35bddba5dfb/users/d2f14bd46a68406aad1bef377a5337f6/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:46:59.762196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28414c9956e43dbb9d6c35bddba5dfb, user_id=d2f14bd46a68406aad1bef377a5337f6, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:46:59.764774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:46:59.791624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:46:59.791624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1089/2186] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:46:59 2022] PUT /identity/v3/projects/e28414c9956e43dbb9d6c35bddba5dfb/users/d2f14bd46a68406aad1bef377a5337f6/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:06.039684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3d23-98f5-4a0c-9c89-98c8f694eab0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:06.040138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3d23-98f5-4a0c-9c89-98c8f694eab0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:06.040363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3d23-98f5-4a0c-9c89-98c8f694eab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:06.077216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a72b3d23-98f5-4a0c-9c89-98c8f694eab0 None None] MFA Rules not processed for user `d2f14bd46a68406aad1bef377a5337f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:06.079495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a72b3d23-98f5-4a0c-9c89-98c8f694eab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:06.174080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1098/2187] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:06.186965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:06.188996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:06.276645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d889c6-2f9b-437d-a7ab-4f32f3d53fa8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:06.280580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:06.281336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:06.281663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:06.282449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:06.375736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:06.377316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:06.394459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1090/2188] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:06.746206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:06.746496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:06.746770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:06.767833 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:06.769085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:06.839924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1099/2189] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:47:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:06.855246 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:06.856730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:06.938521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54139102-e43b-40d7-b26f-3b09fbde608a service glance] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:06.944234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdda985-1dcb-42d7-9d6d-2614c8490f71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:06.945105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdda985-1dcb-42d7-9d6d-2614c8490f71 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:06.945448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdda985-1dcb-42d7-9d6d-2614c8490f71 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:06.945783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdda985-1dcb-42d7-9d6d-2614c8490f71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:06.955047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdda985-1dcb-42d7-9d6d-2614c8490f71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:06.962007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdda985-1dcb-42d7-9d6d-2614c8490f71 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:06.966373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1091/2190] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:47:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:47:08.699648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:08.702564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:08.831078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432247cb-e96d-452f-97ad-6773d2044c21 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:08.837363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:08.838962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:08.839438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:08.839985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] PATH_INFO: `/v3/users/4ac38e3eccb84a6a8a6800ef178a4dc8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:08.841773 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac38e3eccb84a6a8a6800ef178a4dc8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:08.844685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:08.856515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:09.115380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac38e3eccb84a6a8a6800ef178a4dc8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:09.118358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac38e3eccb84a6a8a6800ef178a4dc8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:09.134266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac38e3eccb84a6a8a6800ef178a4dc8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:09.201438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac38e3eccb84a6a8a6800ef178a4dc8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:09.207482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac38e3eccb84a6a8a6800ef178a4dc8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:09.281000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1100/2191] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:47:08 2022] DELETE /identity/v3/users/4ac38e3eccb84a6a8a6800ef178a4dc8 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:09.286262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:09.288410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:09.288708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:09.325490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:09.330386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:09.393597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:09.427484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1092/2192] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:47:09.437870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:09.439534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:09.519325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651133a1-7ac2-4e51-aed9-c6c871a0760c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:09.523844 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c78ce-58e0-4a86-a15a-58fe2b064168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:09.525193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c78ce-58e0-4a86-a15a-58fe2b064168 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:09.525591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c78ce-58e0-4a86-a15a-58fe2b064168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:09.525946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c78ce-58e0-4a86-a15a-58fe2b064168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:09.540697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c78ce-58e0-4a86-a15a-58fe2b064168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:09.543279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037c78ce-58e0-4a86-a15a-58fe2b064168 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:09.559229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1101/2193] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:10.586863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:10.588646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:10.615429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479c55b-cd58-4ab3-b6fd-4b883ab87249 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:10.623686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:10.623686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:10.623970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:10.624157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] PATH_INFO: `/v3/projects/9aa80633bc9c4d98a53a31bd433c45d3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:10.625558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa80633bc9c4d98a53a31bd433c45d3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:10.629680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:10.849033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa80633bc9c4d98a53a31bd433c45d3', 'request_id': 'req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:10.927709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa80633bc9c4d98a53a31bd433c45d3', 'request_id': 'req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:10.935335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa80633bc9c4d98a53a31bd433c45d3', 'request_id': 'req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:10.953259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] The token cache is being invalidate because project 9aa80633bc9c4d98a53a31bd433c45d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:47:10.953830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:10.955390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dd0d8048-d2a5-4c6b-a74d-0a08565771fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:10.959251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1093/2194] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:47:10 2022] DELETE /identity/v3/projects/9aa80633bc9c4d98a53a31bd433c45d3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:47:10.966312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:10.967252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:10.969293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.007622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:11.009259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:11.122289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1102/2195] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:11.130791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.131091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.131368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.151712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:11.153787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:11.167657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1094/2196] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:47:11.171916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:11.182352 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2587166-5603-4356-a0a9-ca883430b411 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:11.186978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:11.187573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.187848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.188116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.188807 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:47:11.189107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:11.191544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:11.203680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1103/2197] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:47:11.208374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:11.218042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fae477-8598-4132-8876-aaf0bae702bc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:11.223738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:11.224436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.224872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.225297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.226427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:11.228800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:11.516773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1095/2198] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:47:11 2022] POST /identity/v3/projects => generated 373 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:11.521452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:11.534025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7da549-e810-48c6-ba94-c8c97959793e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:11.550903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:11.608206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:11.608863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.609146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.609421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.610059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:11.612847 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:11.726656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:11.737777 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1104/2199] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:47:11 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:47:11.743555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf242-6063-4678-8973-20b53874a306 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:11.754311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:11.754947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.755231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.755509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.756426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:11.758596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:11.775185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:11.777459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1096/2200] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:11.791984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84551aef-90fa-44b5-bd11-a31a82c823be admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:11.802076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bb464-3cc1-4684-a2df-571a5fe3d268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:11.806710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb464-3cc1-4684-a2df-571a5fe3d268 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.806710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb464-3cc1-4684-a2df-571a5fe3d268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.806710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bb464-3cc1-4684-a2df-571a5fe3d268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.806710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bb464-3cc1-4684-a2df-571a5fe3d268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:11.807680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bb464-3cc1-4684-a2df-571a5fe3d268 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:11.842017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1105/2201] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:47:11.843128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:11.858125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b0f03-6938-456e-bf41-92f58bd37137 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:11.870865 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:11.871558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:11.871838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:11.872112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] PATH_INFO: `/v3/projects/5825261869ea4295bca2659609c1f4e0/users/72a06a156e0c4c4c820871f5d41ab01c/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:11.908078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5825261869ea4295bca2659609c1f4e0, user_id=72a06a156e0c4c4c820871f5d41ab01c, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:11.915144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:12.025892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:12.026396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:12.026677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:12.029564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1097/2202] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:47:11 2022] PUT /identity/v3/projects/5825261869ea4295bca2659609c1f4e0/users/72a06a156e0c4c4c820871f5d41ab01c/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:12.130314 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] MFA Rules not processed for user `72a06a156e0c4c4c820871f5d41ab01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:12.132084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:12.440732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1106/2203] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:12.446885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:12.450136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:12.565506 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38019926-b470-400e-a17f-676b4f614598 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:12.572588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:12.573472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:12.573761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:12.574134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:12.574507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:12.574905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:12.585344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:12.587085 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:12.599858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1098/2204] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:12.635089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:12.643789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:12.682972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cee39-46a8-4061-a04d-e35c6992f9d6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:12.692357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:12.692995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:12.693266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:12.693554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:12.699626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:12.749487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b00a9-15eb-45dc-b9c5-36c1cbf1a528 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:12.755833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:12.756748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:12.757139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:12.757952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] PATH_INFO: `/v3/users/72a06a156e0c4c4c820871f5d41ab01c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:12.759811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a06a156e0c4c4c820871f5d41ab01c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:12.761992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:12.762122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:12.811340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:12.813002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1c653-1b82-4e0d-b3fd-b79cf75b9992 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:12.835171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1107/2205] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:47:13.009488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a06a156e0c4c4c820871f5d41ab01c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:13.086517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a06a156e0c4c4c820871f5d41ab01c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:13.087169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a06a156e0c4c4c820871f5d41ab01c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:13.104245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a06a156e0c4c4c820871f5d41ab01c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:13.138111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a06a156e0c4c4c820871f5d41ab01c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:13.152814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1099/2206] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:47:12 2022] DELETE /identity/v3/users/72a06a156e0c4c4c820871f5d41ab01c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:47:13.157694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:13.157694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:13.158264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:13.186194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:13.186194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:13.273592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1108/2207] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:47:13.288554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:13.291761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:13.365794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8788b-3332-4dde-89fe-d7a095979315 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:13.370021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cfc11-9605-4613-ba0b-d28e32fdfff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:13.371025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cfc11-9605-4613-ba0b-d28e32fdfff7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:13.371168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cfc11-9605-4613-ba0b-d28e32fdfff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:13.371353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cfc11-9605-4613-ba0b-d28e32fdfff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:13.391896 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cfc11-9605-4613-ba0b-d28e32fdfff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:13.394289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cfc11-9605-4613-ba0b-d28e32fdfff7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:13.412770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1100/2208] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:14.068203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:14.073141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:14.098490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31bd0f-9834-4e77-a7e8-a328d77f2328 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:14.114923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:14.115723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:14.116066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:14.116591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] PATH_INFO: `/v3/projects/5825261869ea4295bca2659609c1f4e0` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:14.117849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5825261869ea4295bca2659609c1f4e0)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:14.136684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:14.358139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5825261869ea4295bca2659609c1f4e0', 'request_id': 'req-43b1ab94-79b0-4a81-b84a-ae229ed619b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:14.366546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5825261869ea4295bca2659609c1f4e0', 'request_id': 'req-43b1ab94-79b0-4a81-b84a-ae229ed619b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:14.469028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5825261869ea4295bca2659609c1f4e0', 'request_id': 'req-43b1ab94-79b0-4a81-b84a-ae229ed619b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:14.498164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] The token cache is being invalidate because project 5825261869ea4295bca2659609c1f4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:47:14.498510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:14.499261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:47:14.502156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1109/2209] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:47:14 2022] DELETE /identity/v3/projects/5825261869ea4295bca2659609c1f4e0 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:47:20.540511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7327506-460e-4076-8602-3aae806786d2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:20.541040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7327506-460e-4076-8602-3aae806786d2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:20.541040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7327506-460e-4076-8602-3aae806786d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:20.560462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e7327506-460e-4076-8602-3aae806786d2 None None] MFA Rules not processed for user `680f726c9e374d97aff5ebe924c84865`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:20.561678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7327506-460e-4076-8602-3aae806786d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:20.580550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:20.623924 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1101/2210] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:20.769199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:20.770720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:20.835730 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:20.850392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1ab94-79b0-4a81-b84a-ae229ed619b8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:20.855909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:20.856670 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:20.856945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:20.857211 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:20.858948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:20.872383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:20.874166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:20.889266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1110/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:47:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:47:24.332716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752de3cb-8874-4802-adc6-966a97440950 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:24.334188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752de3cb-8874-4802-adc6-966a97440950 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:24.334188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752de3cb-8874-4802-adc6-966a97440950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:24.353525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-752de3cb-8874-4802-adc6-966a97440950 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:24.355414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-752de3cb-8874-4802-adc6-966a97440950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:24.420454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1102/2212] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:47:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:24.432486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:24.433899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:24.517087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98e50-d304-47ea-a43b-f6c2b4731e30 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:24.522907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:24.523535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:24.523810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:24.524079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:24.532991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:24.534578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:24.542896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1111/2213] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:47:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:47:34.410676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=39 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:34.539632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=38 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:40.675840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed46892-f89d-49e4-8622-dfc3298aa137 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:40.683405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed46892-f89d-49e4-8622-dfc3298aa137 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:40.683405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed46892-f89d-49e4-8622-dfc3298aa137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:40.699856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4ed46892-f89d-49e4-8622-dfc3298aa137 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:47:40.702497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ed46892-f89d-49e4-8622-dfc3298aa137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:47:40.711636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1103/2214] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:47:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:47:40.729372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:47:40.741916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861b14a-77ad-4825-aa78-eb4efd78276c service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:47:40.747243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88617d-ee6c-4c95-b1c3-45052cea2e41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:47:40.747991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88617d-ee6c-4c95-b1c3-45052cea2e41 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:47:40.748370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88617d-ee6c-4c95-b1c3-45052cea2e41 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:47:40.748706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88617d-ee6c-4c95-b1c3-45052cea2e41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:47:40.758811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88617d-ee6c-4c95-b1c3-45052cea2e41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:47:40.760394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88617d-ee6c-4c95-b1c3-45052cea2e41 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:47:40.769936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1112/2215] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:47:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:47:50.693768 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:47:50.765907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:01.056542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8dbf6-50c1-4ab3-85db-f355ee04ee86 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.057400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8dbf6-50c1-4ab3-85db-f355ee04ee86 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.057400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8dbf6-50c1-4ab3-85db-f355ee04ee86 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.058912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1104/2216] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:48:01.066881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec394-2ab0-4780-a6e0-8149763dd089 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.067445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec394-2ab0-4780-a6e0-8149763dd089 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.067867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec394-2ab0-4780-a6e0-8149763dd089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.093492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a90ec394-2ab0-4780-a6e0-8149763dd089 None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:01.094734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a90ec394-2ab0-4780-a6e0-8149763dd089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.174887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1113/2217] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:01.188102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8dbf6-50c1-4ab3-85db-f355ee04ee86 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:01.190243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3ba8dbf6-50c1-4ab3-85db-f355ee04ee86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.264094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8dbf6-50c1-4ab3-85db-f355ee04ee86 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:01.268128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d920c7-c274-41c2-abf0-40dd55a1a55a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:01.268763 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d920c7-c274-41c2-abf0-40dd55a1a55a service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.269045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d920c7-c274-41c2-abf0-40dd55a1a55a service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.269313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d920c7-c274-41c2-abf0-40dd55a1a55a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.281298 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d920c7-c274-41c2-abf0-40dd55a1a55a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:01.282766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d920c7-c274-41c2-abf0-40dd55a1a55a service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:01.298482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1105/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:01.524734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993072c-2d0b-4dee-9550-d2a9311a142f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.524734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993072c-2d0b-4dee-9550-d2a9311a142f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.525110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993072c-2d0b-4dee-9550-d2a9311a142f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.527088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1114/2219] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 16:48:01.532998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.533279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.533556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.555766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] MFA Rules not processed for user `9bde9ceb3c7c46298626580b62e0ec01`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:01.557990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.628161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1106/2220] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:01.646662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993072c-2d0b-4dee-9550-d2a9311a142f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:01.648971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f993072c-2d0b-4dee-9550-d2a9311a142f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.665228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993072c-2d0b-4dee-9550-d2a9311a142f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:01.669535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678fe6b-c7cf-462c-8ec9-28373181dfe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:01.670419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678fe6b-c7cf-462c-8ec9-28373181dfe2 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.670937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678fe6b-c7cf-462c-8ec9-28373181dfe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.671411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678fe6b-c7cf-462c-8ec9-28373181dfe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.681664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678fe6b-c7cf-462c-8ec9-28373181dfe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:01.683113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678fe6b-c7cf-462c-8ec9-28373181dfe2 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:01.692566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1115/2221] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:48:01.727944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:01.730430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.749707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd04d1a-ee10-4f0c-9273-12a602e20fcc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:01.773231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:01.773977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.774196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.774412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.776631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.799753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca150e47-6085-435a-9945-09330e901147 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.800283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca150e47-6085-435a-9945-09330e901147 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.800657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca150e47-6085-435a-9945-09330e901147 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.802184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1116/2222] 10.222.0.17 () {54 vars in 921 bytes} [Wed Aug 24 16:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 16:48:01.808868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.809143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.809457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.830236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] MFA Rules not processed for user `4542cf167cc24738b441af6530dead3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:01.831440 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:01.841355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1117/2223] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Aug 24 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:01.852272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:01.853315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:01.854332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454d304-1809-4ddd-b102-0dc41ab1de92 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:01.864447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82187e7-cdb1-464f-9c18-6b1b56430fdd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:01.865710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1107/2224] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:01.869402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835575cb-ad4d-4236-9f3b-353f8d68d9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:01.872237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835575cb-ad4d-4236-9f3b-353f8d68d9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:01.872237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835575cb-ad4d-4236-9f3b-353f8d68d9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:01.872237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835575cb-ad4d-4236-9f3b-353f8d68d9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:01.882373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835575cb-ad4d-4236-9f3b-353f8d68d9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:01.883952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835575cb-ad4d-4236-9f3b-353f8d68d9d1 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:01.897358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1118/2225] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:48:07.116444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.117271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dc6ba-3cbe-4146-80fd-30f98633f389 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.117703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.117703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.117848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dc6ba-3cbe-4146-80fd-30f98633f389 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.118073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4dc6ba-3cbe-4146-80fd-30f98633f389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.144263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:07.145982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:07.151030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cf4dc6ba-3cbe-4146-80fd-30f98633f389 None None] MFA Rules not processed for user `1d9f7755863d45bfb04a1498bd62e367`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:07.152679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cf4dc6ba-3cbe-4146-80fd-30f98633f389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:07.173112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1108/2226] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:07.184790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:07.198979 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7934d3-b702-4259-a9d3-8f33babcc558 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:07.204116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f939816-96e4-4145-a7b2-b65599c40982 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:07.204782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f939816-96e4-4145-a7b2-b65599c40982 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:07.205057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f939816-96e4-4145-a7b2-b65599c40982 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:07.205324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f939816-96e4-4145-a7b2-b65599c40982 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:07.214916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f939816-96e4-4145-a7b2-b65599c40982 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:07.216646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f939816-96e4-4145-a7b2-b65599c40982 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:07.225068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1109/2227] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:48:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:48:07.225351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1119/2228] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:17.218081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=71 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:17.221694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:21.873071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:21.874046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:21.874481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:21.899735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] MFA Rules not processed for user `7be50b041b5947d5909ed6ce516ec8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:21.904003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:21.975568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1110/2229] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:48:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:48:30.207501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:30.208030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:30.208030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:30.228206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:48:30.229622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:48:30.237146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1120/2230] 10.222.0.17 () {58 vars in 967 bytes} [Wed Aug 24 16:48:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:48:30.250754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:48:30.263877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f26cc-e422-4cb1-ae91-ff89526dd62c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:48:30.269288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:48:30.269990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:48:30.270343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:48:30.270751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:48:30.283456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:48:30.284856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:48:30.294332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1111/2231] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:48:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:48:40.222973 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:48:40.290367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=31 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:11.359263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:11.360815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:11.419708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c13a9-0879-4925-8e67-bdb695a02eb5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:11.423747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:11.425713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:11.425713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:11.425713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:11.426781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:11.441511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:11.443004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:11.455353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1121/2232] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:14.350065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:14.360481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5924c8f-9ddd-4650-b279-0be6d6c1e059 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:14.365538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:14.365538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:14.365538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:14.365538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:14.367262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:14.382157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:14.383614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:14.395724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1112/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:21.449406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=26 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:24.390176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:26.066860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:26.084286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56759b3-e487-498c-b25e-479cf1ff6c7c service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:26.089654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:26.090474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:26.091025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:26.091511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:26.095871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:26.109536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:26.112309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e550-92da-46b0-91ea-d1c66564e624 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:26.132275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1122/2234] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:30.050815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:30.058737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6afdf7-56bb-4554-92c4-0980b5b397d8 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:30.062531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:30.063774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:30.064031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:30.064305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:30.066427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:30.078605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:30.085901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:30.092020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1113/2235] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:31.310950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.312205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.312471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.347285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] MFA Rules not processed for user `4ce12f8401f3479ba14e5e1c3133a71d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:31.349245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.436790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1123/2236] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:49:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:31.460928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:31.460928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.482214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7743c1-2e7a-4f50-b4b7-11656b5b285a service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:31.494363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:31.495183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:31.495548 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:31.495894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:31.498833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:31.512444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:31.514705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:31.531561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1114/2237] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:32.306950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:32.326491 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2992f-203b-4e0f-b79e-e9c1dd8b3930 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:32.334339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:32.335386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:32.336137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:32.336659 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:32.340280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:32.362231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:32.365412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5ed34-5c93-4e51-8934-d270dc4d779d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:32.383760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1124/2238] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:33.212777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:33.215156 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:33.243426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c320b-5eb8-4a3b-8fbc-723d9e334747 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:33.250687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:33.252027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:33.252487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:33.252775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:33.255342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:33.269866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:33.271553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:33.288481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1115/2239] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:33.542278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:33.543057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:33.543403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:33.603085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] MFA Rules not processed for user `fc27435df46542a3ac37e743b6399387`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:33.606188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:33.723493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1125/2240] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:49:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:49:40.197492 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:40.265229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d159ab-bbed-4103-b90b-0ccf4145db8c service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:40.305478 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:40.306888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.308301 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.311586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.318408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:40.326872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:40.358259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085155d-1296-4567-baf2-35bc152b71a2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:40.359864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:40.363405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:40.390949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:40.394877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:40.398029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:40.398725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:40.404976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1116/2241] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:40.432400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:40.437323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:40.490336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1126/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:44.633197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:44.643808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:44.726655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85803f-bdfb-435c-a16b-82897b3390f6 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:44.743201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:44.747681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:44.748152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:44.748648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:44.751790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:44.775434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:44.777546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:44.795558 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1117/2243] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:47.748955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:47.778422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b7e5c-268f-4043-8c90-281ea8b66228 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:47.795384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:47.796859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:47.800140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:47.800861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:47.806396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:47.842685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:47.846685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca78667-891b-4d11-8183-d829f90e903a service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:47.870303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1127/2244] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:49:49.323118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:49.355095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cdc032-0dbc-4c21-adc1-5f65ffe045bf service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:49.358090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:49.358719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:49.358983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:49.359426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:49.363037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:49.556569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:49.559644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:49.611069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1118/2245] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:49:53.777948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae40da3-c342-41c8-8cc1-896f2f0fb81c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:53.778743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae40da3-c342-41c8-8cc1-896f2f0fb81c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:53.778743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae40da3-c342-41c8-8cc1-896f2f0fb81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:53.789911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=75 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:53.810601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2ae40da3-c342-41c8-8cc1-896f2f0fb81c None None] MFA Rules not processed for user `1d9f7755863d45bfb04a1498bd62e367`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:53.812321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2ae40da3-c342-41c8-8cc1-896f2f0fb81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:53.823229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1128/2246] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:56.303684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:56.305879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:56.314140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=61 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:56.322683 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3926675-205c-43c3-9d69-4862379ea001 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:56.333597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:56.334347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:56.334537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:56.336852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] PATH_INFO: `/v3/users/7be50b041b5947d5909ed6ce516ec8bf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:56.338964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be50b041b5947d5909ed6ce516ec8bf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:56.342872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:56.771310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be50b041b5947d5909ed6ce516ec8bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.907135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be50b041b5947d5909ed6ce516ec8bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.909370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be50b041b5947d5909ed6ce516ec8bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.929218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be50b041b5947d5909ed6ce516ec8bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.949494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be50b041b5947d5909ed6ce516ec8bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:56.959535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1119/2247] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:49:56 2022] DELETE /identity/v3/users/7be50b041b5947d5909ed6ce516ec8bf => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:49:57.494168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.494867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.494867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.520832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:57.522915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:57.626308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.631641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:57.635292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1129/2248] 10.222.0.17 () {58 vars in 967 bytes} [Wed Aug 24 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:57.738359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08058cd5-a03f-4933-af44-56a477b6217d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:57.743971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:57.744952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:57.745432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:57.745709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] PATH_INFO: `/v3/projects/c99d97707cc2485284d8d33522ebb918` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:57.747109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99d97707cc2485284d8d33522ebb918)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:57.750766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:57.965856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:57.968628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:58.008195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99d97707cc2485284d8d33522ebb918', 'request_id': 'req-0073c52b-414e-49be-95e0-0c387a15071d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.089543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537a49d-ffe9-4a7c-8a4d-59764369c97f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:58.212472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19ee52-af89-4ddd-9ef5-4ccbfc71ab00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:58.217772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee52-af89-4ddd-9ef5-4ccbfc71ab00 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.218658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99d97707cc2485284d8d33522ebb918', 'request_id': 'req-0073c52b-414e-49be-95e0-0c387a15071d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.220840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee52-af89-4ddd-9ef5-4ccbfc71ab00 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.221865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee52-af89-4ddd-9ef5-4ccbfc71ab00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.237013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99d97707cc2485284d8d33522ebb918', 'request_id': 'req-0073c52b-414e-49be-95e0-0c387a15071d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.264313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] The token cache is being invalidate because project c99d97707cc2485284d8d33522ebb918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:49:58.265251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.266982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0073c52b-414e-49be-95e0-0c387a15071d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:49:58.275207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1120/2249] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:49:57 2022] DELETE /identity/v3/projects/c99d97707cc2485284d8d33522ebb918 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:58.277573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.278117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.278452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.291653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:58.304973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:58.306593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:58.358219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:49:58.382097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19ee52-af89-4ddd-9ef5-4ccbfc71ab00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:58.385303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19ee52-af89-4ddd-9ef5-4ccbfc71ab00 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:58.396730 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1130/2250] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:49:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:49:58.464679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1121/2251] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:58.475302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.476041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.476798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.526918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:58.528577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:58.542312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1131/2252] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:58.543912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:58.557460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f55e21-111b-400b-adaf-4564453c6661 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:58.563991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:58.565182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.565475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.566611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.567308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:49:58.570171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:58.570171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:58.604438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1122/2253] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:49:58.608894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:58.623816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4ddf8-64a6-4a80-b537-1dce20afdafb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:58.630185 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:58.630818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:58.631292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:58.631735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:58.632587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:58.637855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:58.886804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1132/2254] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:49:58 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:49:58.891391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:58.906602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c32ee6-a990-43c2-a1ca-49b13f4e6eeb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.034284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.035118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.035661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.036319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.037397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.040502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.371753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.374734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1123/2255] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:49:58 2022] POST /identity/v3/users => generated 292 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:49:59.389198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3bc41-136c-4288-b8ee-990b2517a9f1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.398306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.398930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.399300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.399448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.400150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.402047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.425638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.440105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1133/2256] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:49:59.440398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e7e49-9075-4154-8c77-08cece428b5c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.448223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fd4c4-6959-46d7-a38d-15fe37fd4a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.449132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd4c4-6959-46d7-a38d-15fe37fd4a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.449413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd4c4-6959-46d7-a38d-15fe37fd4a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.450788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd4c4-6959-46d7-a38d-15fe37fd4a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.450788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fd4c4-6959-46d7-a38d-15fe37fd4a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.452237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fd4c4-6959-46d7-a38d-15fe37fd4a8d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.470267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1124/2257] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:49:59.479872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.492498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6b16f-71db-4223-9fd2-36dbb6fc0143 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:49:59.498192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:49:59.498930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.499404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.499676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] PATH_INFO: `/v3/projects/28cb03a2e4b24023ba3030a48b882bc8/users/5d36c73bc2444787a5d6e77efca8c16b/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.520409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28cb03a2e4b24023ba3030a48b882bc8, user_id=5d36c73bc2444787a5d6e77efca8c16b, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:49:59.524158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:49:59.785895 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1134/2258] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:49:59 2022] PUT /identity/v3/projects/28cb03a2e4b24023ba3030a48b882bc8/users/5d36c73bc2444787a5d6e77efca8c16b/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:49:59.797147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:49:59.797980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:49:59.798264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:49:59.837197 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] MFA Rules not processed for user `5d36c73bc2444787a5d6e77efca8c16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:49:59.838564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:49:59.953820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1125/2259] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:49:59.967627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:49:59.988729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:00.090784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e15d1-ecc8-4fd3-b571-b7c27139a19e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:00.099001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:00.099767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:00.100221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:00.100677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:00.110376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:00.113205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:00.128778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1135/2260] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:49:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:03.030987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:10.126965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=68,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:10.484740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:10.509804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bf2c5-cf67-43c3-be2b-d5bc385b4761 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:10.516454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:10.517260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:10.517628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:10.519953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:10.520649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:10.540710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:10.542700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:10.575525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1126/2261] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:11.344066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:11.452951 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732183d-5276-41bd-82a8-30192bdcc31b service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:11.461080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:11.461792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:11.462146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:11.462423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] PATH_INFO: `/v3/users/5d36c73bc2444787a5d6e77efca8c16b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:11.463804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d36c73bc2444787a5d6e77efca8c16b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:11.466486 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:11.806206 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:11.809310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:11.831087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687e71c-bbdc-40b1-a9f6-a36a732c77c7 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:11.838390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:11.839121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:11.839563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:11.839862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] PATH_INFO: `/v3/users/680f726c9e374d97aff5ebe924c84865` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:11.841406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680f726c9e374d97aff5ebe924c84865)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:11.843556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:11.872946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d36c73bc2444787a5d6e77efca8c16b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:11.879307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d36c73bc2444787a5d6e77efca8c16b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:11.895916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d36c73bc2444787a5d6e77efca8c16b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.008757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d36c73bc2444787a5d6e77efca8c16b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.009764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ef17448d-2585-4e74-9ab4-8a18757e36de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d36c73bc2444787a5d6e77efca8c16b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.038180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1136/2262] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:11 2022] DELETE /identity/v3/users/5d36c73bc2444787a5d6e77efca8c16b => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:12.042921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:12.043203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:12.044124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:12.066180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:12.067418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:12.116205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680f726c9e374d97aff5ebe924c84865'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.166665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1137/2263] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:12.178365 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:12.180245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:12.210790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:12.231498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680f726c9e374d97aff5ebe924c84865'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.232912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680f726c9e374d97aff5ebe924c84865'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.246259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680f726c9e374d97aff5ebe924c84865'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.255683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51849e-8fb2-4519-8059-746cbb2c313b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:12.260315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:12.261607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:12.262058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:12.262562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:12.266845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680f726c9e374d97aff5ebe924c84865'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:12.274712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1127/2264] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:11 2022] DELETE /identity/v3/users/680f726c9e374d97aff5ebe924c84865 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:12.275500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:12.277057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:12.292479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1138/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:50:12.727326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:12.729350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:12.868394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a75cb-6f00-4b0e-aed5-57157e6192b2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:12.874119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:12.874832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:12.875279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:12.875742 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] PATH_INFO: `/v3/projects/09e2ab2c313d4448a28a6902b74a0d10` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:12.878597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e2ab2c313d4448a28a6902b74a0d10)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:12.881037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:13.063722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:13.065932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:13.079095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e2ab2c313d4448a28a6902b74a0d10', 'request_id': 'req-2783378d-2e85-4e7a-adca-a25fe9ac47dd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.082910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eda7d3-a01a-4966-930f-20fdac8d678d service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:13.156563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:13.163452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:13.163452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:13.163452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] PATH_INFO: `/v3/projects/28cb03a2e4b24023ba3030a48b882bc8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:13.163452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28cb03a2e4b24023ba3030a48b882bc8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:13.163452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:13.234081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e2ab2c313d4448a28a6902b74a0d10', 'request_id': 'req-2783378d-2e85-4e7a-adca-a25fe9ac47dd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.242482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e2ab2c313d4448a28a6902b74a0d10', 'request_id': 'req-2783378d-2e85-4e7a-adca-a25fe9ac47dd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.244104 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:13.255083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] The token cache is being invalidate because project 09e2ab2c313d4448a28a6902b74a0d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:13.255376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.256053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.257577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1128/2266] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:12 2022] DELETE /identity/v3/projects/09e2ab2c313d4448a28a6902b74a0d10 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:13.261600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:13.263004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:13.287023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783378d-2e85-4e7a-adca-a25fe9ac47dd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:13.292045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:13.292754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:13.293033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:13.293310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] PATH_INFO: `/v3/users/d2f14bd46a68406aad1bef377a5337f6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:13.294497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f14bd46a68406aad1bef377a5337f6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:13.296502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:13.393183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28cb03a2e4b24023ba3030a48b882bc8', 'request_id': 'req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.400506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28cb03a2e4b24023ba3030a48b882bc8', 'request_id': 'req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.584528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28cb03a2e4b24023ba3030a48b882bc8', 'request_id': 'req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.597679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] The token cache is being invalidate because project 28cb03a2e4b24023ba3030a48b882bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:13.597679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.598449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b128398d-c4e4-4094-bc4b-e4a4b30d8f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.600130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1139/2267] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:13 2022] DELETE /identity/v3/projects/28cb03a2e4b24023ba3030a48b882bc8 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:13.605278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc9728-b70f-4d95-bdac-aa3d8d273618 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:13.605578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc9728-b70f-4d95-bdac-aa3d8d273618 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:13.605855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc9728-b70f-4d95-bdac-aa3d8d273618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:13.636232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-69fc9728-b70f-4d95-bdac-aa3d8d273618 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:13.637595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-69fc9728-b70f-4d95-bdac-aa3d8d273618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:13.700375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:13.716740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1140/2268] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:13.725462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:13.725783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:13.726100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:13.748192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:13.749682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:13.749829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f14bd46a68406aad1bef377a5337f6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.760418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1141/2269] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:50:13.764684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:13.777225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688265bf-5c47-4417-8c61-c67429d7d2dd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:13.782415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:13.783407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:13.783759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:13.783952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:13.785414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:50:13.785745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:13.787601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:13.804593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1142/2270] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:50:13.810550 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:13.821566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee503ffa-2242-499b-8ca9-b62a97f3247e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:13.826726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:13.827379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:13.827678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:13.828010 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:13.828688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:13.830660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:13.974598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f14bd46a68406aad1bef377a5337f6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.974598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f14bd46a68406aad1bef377a5337f6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.974598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f14bd46a68406aad1bef377a5337f6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.974598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f14bd46a68406aad1bef377a5337f6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:13.979637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1129/2271] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:50:13 2022] DELETE /identity/v3/users/d2f14bd46a68406aad1bef377a5337f6 => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:50:14.219306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1143/2272] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:13 2022] POST /identity/v3/projects => generated 373 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:50:14.224249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:14.225986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:14.314205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee473f-3808-486a-9bab-9ba840f70e8d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:14.390952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:14.397238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:14.397238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:14.397238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:14.397238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:14.397238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:14.450202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:14.454257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:14.471413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441fc54-e0a4-479f-ba80-1e26d341486c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:14.483113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:14.483113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:14.483113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:14.483113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] PATH_INFO: `/v3/projects/e28414c9956e43dbb9d6c35bddba5dfb` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:14.483921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28414c9956e43dbb9d6c35bddba5dfb)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:14.484276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:14.664293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1130/2273] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:14 2022] POST /identity/v3/users => generated 287 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:14.668561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:14.679565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8e8f9-3524-4a18-8258-b003c3e83872 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:14.684353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:14.684959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:14.685232 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:14.685616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:14.686289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:14.687946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:14.698114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1131/2274] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:14.702042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:14.712508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267564a-3c0d-4e05-b346-07b37dbd5eb1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:14.717534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:14.718203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:14.718454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:14.718950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:14.719612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:14.721270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:14.733218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1132/2275] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:50:14.737433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:14.738839 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:14.752714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d0186-7fd9-4ff9-916d-1ef2e6ee138c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:14.776804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28414c9956e43dbb9d6c35bddba5dfb', 'request_id': 'req-ecff34b7-9e39-4db7-97be-97feb47226bc'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:14.784825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28414c9956e43dbb9d6c35bddba5dfb', 'request_id': 'req-ecff34b7-9e39-4db7-97be-97feb47226bc'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:14.820858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:14.821523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:14.821797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:14.822073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] PATH_INFO: `/v3/projects/1b46cf5e97dd441c93392e8195a5261e/users/de1d05d7d3cf482bb95cebe85bc1688d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:14.841719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b46cf5e97dd441c93392e8195a5261e, user_id=de1d05d7d3cf482bb95cebe85bc1688d, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:14.847589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:14.883551 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28414c9956e43dbb9d6c35bddba5dfb', 'request_id': 'req-ecff34b7-9e39-4db7-97be-97feb47226bc'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:14.900094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] The token cache is being invalidate because project e28414c9956e43dbb9d6c35bddba5dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:50:14.900658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:14.901763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ecff34b7-9e39-4db7-97be-97feb47226bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:50:14.904364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1144/2276] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:50:14 2022] DELETE /identity/v3/projects/e28414c9956e43dbb9d6c35bddba5dfb => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:14.995699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1133/2277] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:14 2022] PUT /identity/v3/projects/1b46cf5e97dd441c93392e8195a5261e/users/de1d05d7d3cf482bb95cebe85bc1688d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:15.007134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.007443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.007678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.042218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] MFA Rules not processed for user `de1d05d7d3cf482bb95cebe85bc1688d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:15.043444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:15.130871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1145/2278] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:50:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:15.132365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.134459 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:15.216170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129879e-44a9-4e4c-959d-8aaf11b8b1b0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.231317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.231317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.231317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.231317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.231823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.231823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.379631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1134/2279] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:50:15 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:15.383808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.396368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce93d0-115b-46d8-ba86-f229989f7519 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.418859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:15.465271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.469312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.469312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.469312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.469312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.469312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.553437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1146/2280] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:50:15 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:50:15.558186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.571311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a4779-ebd6-4681-9ab4-1dbab497e6db admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.576764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.577421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.577694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.577960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.578770 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.580449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.592872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1135/2281] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:15.598368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.614376 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009124a4-711d-43c4-8daa-d349da9ce4e6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.621774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.622929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.623430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.623942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] PATH_INFO: `/v3/projects/f2a56d40e8c1449eb0627dcf8c2f1a5e/users/af15acf0834c4731b45559769d8b02d4/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.650726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a56d40e8c1449eb0627dcf8c2f1a5e, user_id=af15acf0834c4731b45559769d8b02d4, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.654411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.748619 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1147/2282] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:15 2022] PUT /identity/v3/projects/f2a56d40e8c1449eb0627dcf8c2f1a5e/users/af15acf0834c4731b45559769d8b02d4/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:15.754960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.766569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17a270-82a9-43e0-b397-66a8bec306a1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.802657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:15.833471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9e490-431e-4890-ba64-7d474a1c0c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.834354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9e490-431e-4890-ba64-7d474a1c0c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.835078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9e490-431e-4890-ba64-7d474a1c0c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.835454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9e490-431e-4890-ba64-7d474a1c0c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.836216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9e490-431e-4890-ba64-7d474a1c0c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.838077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9e490-431e-4890-ba64-7d474a1c0c55 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.850707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1136/2283] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:50:15.854332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:15.866507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b17b08-8ded-4e21-964d-144d3ecd7818 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:15.872830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:15.873507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.873785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.874061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] PATH_INFO: `/v3/projects/f2a56d40e8c1449eb0627dcf8c2f1a5e/users/af15acf0834c4731b45559769d8b02d4/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.876071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a56d40e8c1449eb0627dcf8c2f1a5e, user_id=af15acf0834c4731b45559769d8b02d4, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:15.880069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:15.896110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1148/2284] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:50:15 2022] PUT /identity/v3/projects/f2a56d40e8c1449eb0627dcf8c2f1a5e/users/af15acf0834c4731b45559769d8b02d4/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:50:15.907587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:15.908429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:15.908795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:15.945527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] MFA Rules not processed for user `af15acf0834c4731b45559769d8b02d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:50:15.947100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:16.030454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1137/2285] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:50:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:50:16.048068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:16.050445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:16.120899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b59c2-1976-45b1-8ae5-ecf5f3a7d4dc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:16.124947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:16.125507 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:16.125786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:16.126052 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:16.141208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:16.149226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:16.154079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1149/2286] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:16.400564 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:16.411281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7379521-2127-421d-ab65-81ab22684a3a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:16.414861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40785930-7934-47fa-a636-c9c5fe1a52c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:16.415476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785930-7934-47fa-a636-c9c5fe1a52c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:16.415747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785930-7934-47fa-a636-c9c5fe1a52c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:16.416010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785930-7934-47fa-a636-c9c5fe1a52c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:16.481654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785930-7934-47fa-a636-c9c5fe1a52c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:16.483127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785930-7934-47fa-a636-c9c5fe1a52c1 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:16.493677 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1138/2287] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:26.149410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=59,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:26.489606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=76 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:50:44.591649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:50:44.593069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:44.605070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452cd63-c975-4590-beed-c4cfc7bf9fd1 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:50:44.610680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:50:44.611294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:50:44.611565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:50:44.611828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:50:44.614409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:50:44.683911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:50:44.685387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:50:44.694742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1150/2288] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:50:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:50:54.690187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=28 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:09.016017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:09.016580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:09.016778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:09.037327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:09.038593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:09.097279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1139/2289] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:51:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:09.327707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:09.329150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:09.399612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75275f0-786e-4738-92ba-a3026cd8914a service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:09.405037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd518f-f26e-4501-88a1-5218f78ee242 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:09.405665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd518f-f26e-4501-88a1-5218f78ee242 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:09.405953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd518f-f26e-4501-88a1-5218f78ee242 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:09.406555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd518f-f26e-4501-88a1-5218f78ee242 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:09.415210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd518f-f26e-4501-88a1-5218f78ee242 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:09.416829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd518f-f26e-4501-88a1-5218f78ee242 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:09.425007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1151/2290] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:51:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 16:51:10.660531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:10.874328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb3ede-0608-4c56-b3d1-58f2ee7d5639 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:10.881950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:10.883247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:10.885889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:10.885889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] PATH_INFO: `/v3/users/de1d05d7d3cf482bb95cebe85bc1688d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:10.890862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1d05d7d3cf482bb95cebe85bc1688d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:10.890862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:11.144151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1d05d7d3cf482bb95cebe85bc1688d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:11.254408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1d05d7d3cf482bb95cebe85bc1688d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:11.255974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1d05d7d3cf482bb95cebe85bc1688d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:11.269514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1d05d7d3cf482bb95cebe85bc1688d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:11.301733 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1d05d7d3cf482bb95cebe85bc1688d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:11.314878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1140/2291] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:10 2022] DELETE /identity/v3/users/de1d05d7d3cf482bb95cebe85bc1688d => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:11.318198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:11.318825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:11.319146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:11.348282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:11.350291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:11.468243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1152/2292] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:11.481325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:11.483475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:11.543507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:11.593042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7404c2-5487-46b1-9acc-241e486d9b62 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:11.601886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:11.603756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:11.603756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:11.603959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:11.632445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:11.634035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:11.651837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1141/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:12.198034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:12.199527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:12.217740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87f1c6-3f9f-445f-972c-7270eb8e6646 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:12.233375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:12.236188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:12.236977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:12.237501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] PATH_INFO: `/v3/projects/1b46cf5e97dd441c93392e8195a5261e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:12.240678 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b46cf5e97dd441c93392e8195a5261e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:12.281114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:12.557801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b46cf5e97dd441c93392e8195a5261e', 'request_id': 'req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:12.589933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b46cf5e97dd441c93392e8195a5261e', 'request_id': 'req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:12.986203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b46cf5e97dd441c93392e8195a5261e', 'request_id': 'req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:12.998845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] The token cache is being invalidate because project 1b46cf5e97dd441c93392e8195a5261e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:12.999118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:12.999804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:13.001637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1153/2294] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:12 2022] DELETE /identity/v3/projects/1b46cf5e97dd441c93392e8195a5261e => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:13.007239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:13.025430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e057929-b43c-4812-a308-665fcfe98578 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:13.110658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:13.111745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:13.112224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:13.112806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] PATH_INFO: `/v3/users/af15acf0834c4731b45559769d8b02d4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:13.115471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=af15acf0834c4731b45559769d8b02d4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:13.117929 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:13.384583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af15acf0834c4731b45559769d8b02d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:13.505776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af15acf0834c4731b45559769d8b02d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:13.507460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af15acf0834c4731b45559769d8b02d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:13.522495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af15acf0834c4731b45559769d8b02d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:13.549671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-948765be-ce45-4e9f-821a-fb1ca073d0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af15acf0834c4731b45559769d8b02d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:13.558004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1142/2295] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:13 2022] DELETE /identity/v3/users/af15acf0834c4731b45559769d8b02d4 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:14.444018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:14.446004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:14.491727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e05bc7-d8e3-4e02-9615-f36dd3bde0e9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:14.510727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:14.582244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:14.583396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:14.583870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:14.584147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] PATH_INFO: `/v3/projects/f2a56d40e8c1449eb0627dcf8c2f1a5e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:14.585343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a56d40e8c1449eb0627dcf8c2f1a5e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:14.587531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:14.728417 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a56d40e8c1449eb0627dcf8c2f1a5e', 'request_id': 'req-e265ea87-c510-4224-abe5-aea0c11a822d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:14.751149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a56d40e8c1449eb0627dcf8c2f1a5e', 'request_id': 'req-e265ea87-c510-4224-abe5-aea0c11a822d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:14.871549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a56d40e8c1449eb0627dcf8c2f1a5e', 'request_id': 'req-e265ea87-c510-4224-abe5-aea0c11a822d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:14.887451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] The token cache is being invalidate because project f2a56d40e8c1449eb0627dcf8c2f1a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:14.887981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:14.891740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e265ea87-c510-4224-abe5-aea0c11a822d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:14.893552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1154/2296] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:14 2022] DELETE /identity/v3/projects/f2a56d40e8c1449eb0627dcf8c2f1a5e => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:14.900530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:14.900530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:14.900530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:14.931255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:14.935119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:15.015998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:15.060891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1143/2297] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:15.065068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.074180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.074180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.116679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:15.118738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:15.131166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1155/2298] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:15.137751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:15.149965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac66bb-46ca-47a1-afcb-30c6af5093a0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:15.182722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:15.183537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.183825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.184101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.184830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:15.185137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:15.187160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:15.210466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1144/2299] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:15.235814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:15.246878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a37e0-0e66-4f3d-9249-821a77900186 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:15.256401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:15.260547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.261072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.261459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.262285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:15.264380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:15.372966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1156/2300] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:15 2022] POST /identity/v3/projects => generated 407 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:15.378928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:15.397114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001965a-6de3-4b0c-aee9-0b732ca4a0ff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:15.540704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:15.541539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.542135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.542430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.543776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:15.546390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:15.773136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1145/2301] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:15 2022] POST /identity/v3/users => generated 304 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:15.777109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:15.797481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d2664-1418-4b7c-9360-220b607e512f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:15.813128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:15.813828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.814150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.814523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.815472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:15.818366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:15.832375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1157/2302] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:15.837524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:15.853015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce1386-9016-42ba-9e40-3604f41d4e84 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:15.859107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65134ad-ddc7-41fb-8178-cb03ee2a4724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:15.859952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65134ad-ddc7-41fb-8178-cb03ee2a4724 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.860283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65134ad-ddc7-41fb-8178-cb03ee2a4724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.860603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65134ad-ddc7-41fb-8178-cb03ee2a4724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.861279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65134ad-ddc7-41fb-8178-cb03ee2a4724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:15.862982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65134ad-ddc7-41fb-8178-cb03ee2a4724 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:15.882687 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1146/2303] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:51:15.887504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:15.907839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3611e0-96dd-4af7-bcc4-749537a31576 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:15.917643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:15.918326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:15.918607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:15.918878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] PATH_INFO: `/v3/projects/e300a5dbc332469d88f242af2ef33521/users/99849954479e48f9a9a4bf39dc148695/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:15.959389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e300a5dbc332469d88f242af2ef33521, user_id=99849954479e48f9a9a4bf39dc148695, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:15.967704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:16.081043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1158/2304] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:15 2022] PUT /identity/v3/projects/e300a5dbc332469d88f242af2ef33521/users/99849954479e48f9a9a4bf39dc148695/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:16.094580 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:16.095310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:16.095310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:16.138909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] MFA Rules not processed for user `99849954479e48f9a9a4bf39dc148695`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:16.140993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:16.265311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1147/2305] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:16 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:16.271163 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:16.355864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:16.385249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66b1dd-7ba1-401f-9117-efcc234f21e2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:16.398634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:16.426045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:16.426045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:16.426045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:16.426045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:16.426045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:16.870065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1159/2306] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:16 2022] POST /identity/v3/projects => generated 409 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:16.871895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:16.885951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ee586-988e-4f96-8c43-2b7bcf697912 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:16.937008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:16.955245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:16.964959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:16.964959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:16.964959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:16.964959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:16.964959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:17.130392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1148/2307] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:16 2022] POST /identity/v3/users => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:17.136143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:17.155047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddd9a4-8dcc-487d-b417-7244453ff788 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:17.163542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:17.164666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:17.165164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:17.165649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:17.166892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:17.170306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:17.192482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1160/2308] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:17.194513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:17.206096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d39fa4-789a-43c6-b4fe-a888c5031d44 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:17.213144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:17.220673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:17.220673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:17.220673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] PATH_INFO: `/v3/projects/60377a5fa4fc4e0b99c2612d9d243e09/users/e672772611e14742a10f2c86a20273a6/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:17.244773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60377a5fa4fc4e0b99c2612d9d243e09, user_id=e672772611e14742a10f2c86a20273a6, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:17.249657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:17.375978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1149/2309] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:17 2022] PUT /identity/v3/projects/60377a5fa4fc4e0b99c2612d9d243e09/users/e672772611e14742a10f2c86a20273a6/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:17.380956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:17.403960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd7e08-398a-4fd2-956c-4280b564c3ae admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:17.495129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd21ed-ee75-4441-978b-8cec4142aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:17.495129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd21ed-ee75-4441-978b-8cec4142aae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:17.495542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd21ed-ee75-4441-978b-8cec4142aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:17.510408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd21ed-ee75-4441-978b-8cec4142aae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:17.516786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd21ed-ee75-4441-978b-8cec4142aae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:17.529828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd21ed-ee75-4441-978b-8cec4142aae9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:17.544152 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1161/2310] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:17.548967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:17.579552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1264b1-594b-44ec-b90e-6206cda25568 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:17.590286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:17.591877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:17.591877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:17.592086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] PATH_INFO: `/v3/projects/60377a5fa4fc4e0b99c2612d9d243e09/users/e672772611e14742a10f2c86a20273a6/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:17.596908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60377a5fa4fc4e0b99c2612d9d243e09, user_id=e672772611e14742a10f2c86a20273a6, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:17.603745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:17.625866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1150/2311] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:17 2022] PUT /identity/v3/projects/60377a5fa4fc4e0b99c2612d9d243e09/users/e672772611e14742a10f2c86a20273a6/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:17.637786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:17.637786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:17.638224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:17.694093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] MFA Rules not processed for user `e672772611e14742a10f2c86a20273a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:17.695868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:17.815124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1162/2312] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:17 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:17.829425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:17.831723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:18.029437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eb632-ec1d-4665-a40f-07b5ba1271f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:18.042657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:18.043723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:18.044446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:18.045045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:18.081116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:18.084495 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:18.119349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1151/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:17 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:18.921324 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:18.936388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a0a98-07cb-4819-a6d8-5baf85311956 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:18.946191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb0ed4-81d5-414e-9d2f-59f6c96bd679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:18.948459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb0ed4-81d5-414e-9d2f-59f6c96bd679 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:18.948459 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb0ed4-81d5-414e-9d2f-59f6c96bd679 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:18.949232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb0ed4-81d5-414e-9d2f-59f6c96bd679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:19.035133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:19.101877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb0ed4-81d5-414e-9d2f-59f6c96bd679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:19.104384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb0ed4-81d5-414e-9d2f-59f6c96bd679 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:19.123026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1163/2314] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:18 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:51:28.096816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=51,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:29.148982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=12 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:32.168778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:32.256653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac947333-e411-4420-8cb2-04e1a0bb4bdc service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:32.261690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:32.262322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:32.262599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:32.262869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] PATH_INFO: `/v3/users/99849954479e48f9a9a4bf39dc148695` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:32.264017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99849954479e48f9a9a4bf39dc148695)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:32.266209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:32.480812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99849954479e48f9a9a4bf39dc148695'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:32.586377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99849954479e48f9a9a4bf39dc148695'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:32.587092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99849954479e48f9a9a4bf39dc148695'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:32.594850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99849954479e48f9a9a4bf39dc148695'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:32.608874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99849954479e48f9a9a4bf39dc148695'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:32.626537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1152/2315] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:32 2022] DELETE /identity/v3/users/99849954479e48f9a9a4bf39dc148695 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:32.626703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:32.626703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:32.626703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:32.644257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:32.645545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:32.712468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1164/2316] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:32.719442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:32.721343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:32.784611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55dc06-94c3-4f2d-b94f-9191cb63a8f6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:32.790126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:32.791101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:32.791378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:32.791647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:32.801665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:32.803633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:32.813423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1153/2317] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:33.275245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:33.276736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:33.292437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ffd9-60d3-40f1-b387-fc85ff106ff6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:33.298191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:33.298843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:33.299118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:33.299397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] PATH_INFO: `/v3/projects/e300a5dbc332469d88f242af2ef33521` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:33.301058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e300a5dbc332469d88f242af2ef33521)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:33.303845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:33.481924 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e300a5dbc332469d88f242af2ef33521', 'request_id': 'req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:33.491821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e300a5dbc332469d88f242af2ef33521', 'request_id': 'req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:33.597484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e300a5dbc332469d88f242af2ef33521', 'request_id': 'req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:33.609554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] The token cache is being invalidate because project e300a5dbc332469d88f242af2ef33521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:33.609952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:33.610660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:33.612046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1165/2318] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:33 2022] DELETE /identity/v3/projects/e300a5dbc332469d88f242af2ef33521 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:33.616101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:33.626036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4ddd7-fcaa-4c5d-a3f3-5f3775f304c9 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:33.644960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:33.683904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:33.684479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:33.684755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:33.685062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] PATH_INFO: `/v3/users/e672772611e14742a10f2c86a20273a6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:33.686758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e672772611e14742a10f2c86a20273a6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:33.688856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:33.944947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e672772611e14742a10f2c86a20273a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:34.079179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e672772611e14742a10f2c86a20273a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:34.079831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e672772611e14742a10f2c86a20273a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:34.090163 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e672772611e14742a10f2c86a20273a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:34.106385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d4a3e3e-2d7a-4a35-afbb-fb496f946cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e672772611e14742a10f2c86a20273a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:34.115178 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1154/2319] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:33 2022] DELETE /identity/v3/users/e672772611e14742a10f2c86a20273a6 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:34.665976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:34.668149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:34.700191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998d2a-7ec2-4b75-9fd5-fe89d6f5dedb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:34.783071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:34.783716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:34.783992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:34.784262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] PATH_INFO: `/v3/projects/60377a5fa4fc4e0b99c2612d9d243e09` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:34.787019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=60377a5fa4fc4e0b99c2612d9d243e09)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:34.789281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:34.909866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:34.923325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60377a5fa4fc4e0b99c2612d9d243e09', 'request_id': 'req-1debebfa-c739-4fdd-9d7e-39af9243c14f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:34.931400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60377a5fa4fc4e0b99c2612d9d243e09', 'request_id': 'req-1debebfa-c739-4fdd-9d7e-39af9243c14f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:35.049112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60377a5fa4fc4e0b99c2612d9d243e09', 'request_id': 'req-1debebfa-c739-4fdd-9d7e-39af9243c14f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:35.065845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] The token cache is being invalidate because project 60377a5fa4fc4e0b99c2612d9d243e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:35.066851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:35.079418 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1debebfa-c739-4fdd-9d7e-39af9243c14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:35.081119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1166/2320] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:34 2022] DELETE /identity/v3/projects/60377a5fa4fc4e0b99c2612d9d243e09 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:35.086923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.087473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.087473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.109697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:35.111803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:35.182155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1155/2321] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:35.190287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.190572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.190880 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.214549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:35.215783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:35.225181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1167/2322] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:35.237960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:35.254560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779c355-ed27-4c67-aeb5-ab1e22e399f1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:35.267649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:35.268878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.269252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.269437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.270099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:35.270395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:35.272236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:35.298184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1156/2323] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:35.306116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:35.316769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed85791-3c0c-4033-88ed-7ce82ce85dd6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:35.321975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:35.323042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.323385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.325442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.327820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:35.329918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:35.498858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1168/2324] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:35 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:35.505257 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:35.521715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a4d8c-394c-40da-a302-dd6ecb369b01 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:35.552546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:35.586224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:35.587461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.588725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.589021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.589695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:35.591724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:35.762406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1157/2325] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:35 2022] POST /identity/v3/users => generated 291 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:35.766572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:35.778892 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b51a8f-17ab-4ff6-baa9-eb3dcdef2015 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:35.783586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:35.784252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.784573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.784838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.785606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:35.788128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:35.799330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1169/2326] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:51:35.803681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:35.814119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794d49f-d406-428d-88c2-28e2b05da161 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:35.819978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de47f9e-250d-4dbf-8201-6155610bc874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:35.821446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de47f9e-250d-4dbf-8201-6155610bc874 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.821446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de47f9e-250d-4dbf-8201-6155610bc874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.821446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de47f9e-250d-4dbf-8201-6155610bc874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.822026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de47f9e-250d-4dbf-8201-6155610bc874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:35.824014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de47f9e-250d-4dbf-8201-6155610bc874 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:35.837380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1158/2327] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:35.843658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:35.866239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376808d7-a185-4600-be9b-c549fb2cef66 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:35.888781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:35.888781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:35.888781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:35.888781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] PATH_INFO: `/v3/projects/b152433ebf0249c990aa97eea4354e88/users/d22b131cc0864bfa88516768e54a66d5/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:35.921271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b152433ebf0249c990aa97eea4354e88, user_id=d22b131cc0864bfa88516768e54a66d5, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:35.930053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:36.136803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1170/2328] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:35 2022] PUT /identity/v3/projects/b152433ebf0249c990aa97eea4354e88/users/d22b131cc0864bfa88516768e54a66d5/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:36.147542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:36.148395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:36.148596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:36.217999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] MFA Rules not processed for user `d22b131cc0864bfa88516768e54a66d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:36.219966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:36.378035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1159/2329] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:36.383886 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:36.557576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d9571-0786-4237-9bd8-508d51e57856 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:36.574343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:36.576058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:36.576567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:36.577285 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:36.578388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:36.581976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:36.773542 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1171/2330] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:36 2022] POST /identity/v3/projects => generated 381 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:51:36.778370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:36.790571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9438b-5b9e-4246-8a98-460db9ee50ef None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:36.861912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:36.862732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:36.863165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:36.863437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:36.864127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:36.866401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:37.004137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1160/2331] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:36 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:37.005998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:37.017672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae3db7-1942-4d0d-8c10-0b0b9ece9b77 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:37.024323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:37.025851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.025851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.026128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.026788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:37.028577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:37.045579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1172/2332] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:51:37.054158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:37.066590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b956f-87a1-45fd-b613-00d9e3355d12 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:37.073661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:37.087641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.087641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.087963 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] PATH_INFO: `/v3/projects/19486db007ad4b97ad61b6c4c839fb44/users/719cde615cea4779aa6a1b9a577fba9a/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.148445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19486db007ad4b97ad61b6c4c839fb44, user_id=719cde615cea4779aa6a1b9a577fba9a, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:37.151343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:37.259339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1161/2333] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:37 2022] PUT /identity/v3/projects/19486db007ad4b97ad61b6c4c839fb44/users/719cde615cea4779aa6a1b9a577fba9a/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:37.263325 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:37.274690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ae752-6669-4bf2-8cf3-f561aab6be02 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:37.303213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:37.356996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8a831-b082-46b4-bc85-647227c8f0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:37.357905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a831-b082-46b4-bc85-647227c8f0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.358514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a831-b082-46b4-bc85-647227c8f0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.358802 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a831-b082-46b4-bc85-647227c8f0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.359538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8a831-b082-46b4-bc85-647227c8f0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:37.361222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8a831-b082-46b4-bc85-647227c8f0b9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:37.379485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1173/2334] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:37.380927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:37.387586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:37.393028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7911c-7e92-47cb-9f51-db7b641d1fa4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:37.399286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:37.400598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.400925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.405641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] PATH_INFO: `/v3/projects/19486db007ad4b97ad61b6c4c839fb44/users/719cde615cea4779aa6a1b9a577fba9a/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.407626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=19486db007ad4b97ad61b6c4c839fb44, user_id=719cde615cea4779aa6a1b9a577fba9a, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:37.410826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:37.429509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1162/2335] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:37 2022] PUT /identity/v3/projects/19486db007ad4b97ad61b6c4c839fb44/users/719cde615cea4779aa6a1b9a577fba9a/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:37.439623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.440257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.441030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.549887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] MFA Rules not processed for user `719cde615cea4779aa6a1b9a577fba9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:37.551300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:37.706714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1174/2336] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:37.718833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:37.722439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:37.810141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d98c64-9709-495a-b1bb-5b9e65e0c6ae admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:37.814784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392ba57-f1c1-4269-8c29-7ff29b5cb0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:37.815493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ba57-f1c1-4269-8c29-7ff29b5cb0dd service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:37.815778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ba57-f1c1-4269-8c29-7ff29b5cb0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:37.816048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ba57-f1c1-4269-8c29-7ff29b5cb0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:37.834227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392ba57-f1c1-4269-8c29-7ff29b5cb0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:37.835905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392ba57-f1c1-4269-8c29-7ff29b5cb0dd service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:37.848486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1163/2337] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:38.089318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:38.101629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a89ca-e2b3-472e-baa9-f18ec83bf749 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:38.106733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:38.107236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:38.107516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:38.107787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:38.174672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:38.176146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:38.185769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1175/2338] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:38.401990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:38.402616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:38.402616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:38.426956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:38.428854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:38.489661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1164/2339] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:51:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:38.502202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:38.503604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:38.556958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:38.594615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7e8f0-cecc-4f23-995a-82b8e38685a2 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:38.624746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e287461-a0f6-4e1b-8f82-9dff5ab20c5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:38.626145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e287461-a0f6-4e1b-8f82-9dff5ab20c5f service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:38.626658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e287461-a0f6-4e1b-8f82-9dff5ab20c5f service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:38.627110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e287461-a0f6-4e1b-8f82-9dff5ab20c5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:38.646741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e287461-a0f6-4e1b-8f82-9dff5ab20c5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:38.652922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e287461-a0f6-4e1b-8f82-9dff5ab20c5f service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:38.671688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1176/2340] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:51:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:51:38.745027 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:38.839824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba800f-bdb6-4ae6-8708-135612c6f4d4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:38.846055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:38.846895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:38.847561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:38.848053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] PATH_INFO: `/v3/users/d22b131cc0864bfa88516768e54a66d5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:38.849471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22b131cc0864bfa88516768e54a66d5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:38.851826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:39.145308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:39.169293 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22b131cc0864bfa88516768e54a66d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:39.259234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22b131cc0864bfa88516768e54a66d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:39.260266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22b131cc0864bfa88516768e54a66d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:39.271448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22b131cc0864bfa88516768e54a66d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:39.311852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22b131cc0864bfa88516768e54a66d5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:39.327710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1165/2341] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:38 2022] DELETE /identity/v3/users/d22b131cc0864bfa88516768e54a66d5 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:39.329603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:39.330433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:39.331882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:39.382595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:39.390410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:39.557207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1177/2342] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:39.576464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:39.578639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:39.684192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ab6f3-7a20-41fe-bde6-27a70ba0d8a9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:39.689408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:39.690515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:39.690936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:39.691297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:39.707691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:39.710613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:39.735698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1166/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:41.094581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:41.096868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:41.139364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9441b-c0c9-4414-9290-68bca6b94941 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:41.157233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:41.157233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:41.157233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:41.157233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] PATH_INFO: `/v3/projects/b152433ebf0249c990aa97eea4354e88` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:41.158070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b152433ebf0249c990aa97eea4354e88)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:41.158070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:41.294763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b152433ebf0249c990aa97eea4354e88', 'request_id': 'req-dedcec96-2178-4d23-af48-96e359153865'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.303495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b152433ebf0249c990aa97eea4354e88', 'request_id': 'req-dedcec96-2178-4d23-af48-96e359153865'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.364512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b152433ebf0249c990aa97eea4354e88', 'request_id': 'req-dedcec96-2178-4d23-af48-96e359153865'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.387105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] The token cache is being invalidate because project b152433ebf0249c990aa97eea4354e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:41.387461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.388249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.389834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1178/2344] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:41 2022] DELETE /identity/v3/projects/b152433ebf0249c990aa97eea4354e88 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:41.394013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:41.405077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41900274-60b8-4dc6-ae3b-7308903ffdd5 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:41.568193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:41.569787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:41.570077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:41.582966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] PATH_INFO: `/v3/users/719cde615cea4779aa6a1b9a577fba9a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:41.589185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] RBAC: Authorizing `identity:delete_user(user_id=719cde615cea4779aa6a1b9a577fba9a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:41.591219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:41.804661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719cde615cea4779aa6a1b9a577fba9a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.878154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719cde615cea4779aa6a1b9a577fba9a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.878900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719cde615cea4779aa6a1b9a577fba9a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.890913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719cde615cea4779aa6a1b9a577fba9a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.892630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:41.920099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-41ab1205-99c1-45d6-9748-25fe2373413c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719cde615cea4779aa6a1b9a577fba9a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:41.928592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1167/2345] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:41 2022] DELETE /identity/v3/users/719cde615cea4779aa6a1b9a577fba9a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:42.565801 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:42.569411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:42.605138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedcec96-2178-4d23-af48-96e359153865 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:42.696589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:42.721302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:42.722446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:42.722654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:42.741074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] PATH_INFO: `/v3/projects/19486db007ad4b97ad61b6c4c839fb44` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:42.741074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19486db007ad4b97ad61b6c4c839fb44)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:42.741074 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:42.911719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19486db007ad4b97ad61b6c4c839fb44', 'request_id': 'req-bc222dcc-145c-45bc-b912-083107fe4908'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:42.944897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19486db007ad4b97ad61b6c4c839fb44', 'request_id': 'req-bc222dcc-145c-45bc-b912-083107fe4908'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:43.045064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19486db007ad4b97ad61b6c4c839fb44', 'request_id': 'req-bc222dcc-145c-45bc-b912-083107fe4908'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:43.083617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] The token cache is being invalidate because project 19486db007ad4b97ad61b6c4c839fb44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:43.083617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:43.083617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bc222dcc-145c-45bc-b912-083107fe4908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:43.083617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1179/2346] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:42 2022] DELETE /identity/v3/projects/19486db007ad4b97ad61b6c4c839fb44 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:43.092930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:43.093418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:43.096210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:43.150961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:43.152957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:43.314127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1168/2347] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:43.317389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:43.317915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:43.318499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:43.355030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:43.357813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:43.386002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1180/2348] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:43.390932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:43.410999 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9c91d-03bd-4a51-9aef-e5bba52a02f2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:43.428209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:43.429073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:43.429516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:43.430018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:43.433263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:43.434075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:43.436439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:43.472111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1169/2349] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:43.477589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:43.487776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4b530-b314-42c0-a34b-44916c88caa1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:43.493942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:43.494842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:43.495099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:43.495375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:43.496103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:43.498222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:43.714222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1181/2350] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:43 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:51:43.719637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:43.749187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad795878-0e33-44bf-8f1a-bf541768c651 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:43.810051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:43.812306 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:43.939644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:43.941302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:43.942128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:43.943086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:43.945155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:43.948243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:43.981489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc938-68d7-4657-b379-65542513eb0f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:43.987342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:43.988428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:43.988808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:43.989170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:43.992214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:44.095397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:44.097424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:44.111139 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1182/2351] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:44.111614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1170/2352] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:43 2022] POST /identity/v3/users => generated 289 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:44.118271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:44.138768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb05f80-69c4-4ba5-9795-247627dc32bf service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:44.154122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:44.154832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:44.155115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:44.155404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:44.156124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:44.157926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:44.185561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1183/2353] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:44.190381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:44.230374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5576151-d988-4fe8-8bfc-25199173fab1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:44.237226 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf5b46-56fe-4594-b6a3-3a62bd30dffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:44.238228 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf5b46-56fe-4594-b6a3-3a62bd30dffe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:44.238732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf5b46-56fe-4594-b6a3-3a62bd30dffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:44.239076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf5b46-56fe-4594-b6a3-3a62bd30dffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:44.240041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf5b46-56fe-4594-b6a3-3a62bd30dffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:44.243421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf5b46-56fe-4594-b6a3-3a62bd30dffe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:44.265698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1171/2354] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:44.270406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:44.279754 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:44.290848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabc2bc-af44-4979-9a8f-9059f1c2b1d8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:44.297888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:44.298927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:44.299527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:44.300075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] PATH_INFO: `/v3/projects/758d557cade148a291927dbdda413ad6/users/b94aa32205404578afd23f6bc4f8c14d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:44.324355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758d557cade148a291927dbdda413ad6, user_id=b94aa32205404578afd23f6bc4f8c14d, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:44.329339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:44.582873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1184/2355] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:44 2022] PUT /identity/v3/projects/758d557cade148a291927dbdda413ad6/users/b94aa32205404578afd23f6bc4f8c14d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:44.595191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:44.595846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:44.596006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:44.638137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] MFA Rules not processed for user `b94aa32205404578afd23f6bc4f8c14d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:44.639676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:44.704132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:44.786328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1172/2356] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:44.805240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:44.808394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:44.952250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea1cca-ce10-4513-8f75-baf65a955de0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:44.957574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:44.964202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:44.964202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:44.964202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:44.979498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:44.981224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:44.994795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1185/2357] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:45.137950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:45.140690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:45.179043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:45.182405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:45.319709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d981898-3111-45ce-86cf-eff4cfa1bcaa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:45.331260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:45.332695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:45.333635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:45.334751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:45.339957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:45.389775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e9b56-8ed9-4913-b693-ab36835f12fb service glance] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:45.403697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:45.404618 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:45.405266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:45.405645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:45.411208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:45.464128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:45.465567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:45.488953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1173/2358] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:45.594065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:45.596449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:45.627454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1186/2359] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:55.616956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:55.673538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=62 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:56.327203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:56.351299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e70c8-8a44-4231-ac5e-4e8dc620a658 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:56.362057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:56.367264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:56.367264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:56.367264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:56.368694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:56.401114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:56.407406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:56.433725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1174/2360] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:56.466831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:56.469179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:56.599407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9af311-67e0-4790-8ef5-ad1f15036771 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:56.611504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:56.611858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:56.612911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:56.613143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:56.613584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:56.615387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:56.619689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:56.658891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:56.660854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cd195-bcec-4101-a204-017c417dbd3b service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:56.661177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:56.681656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1187/2361] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:51:56.681970 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:56.683146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:56.683530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:56.683817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:56.684241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:56.686056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:56.688264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:56.715178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f76bb5-8799-4280-9db4-7e12a870d499 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:56.721948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:56.722583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:56.722859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:56.723154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] PATH_INFO: `/v3/users/1d9f7755863d45bfb04a1498bd62e367` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:56.724527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:56.724925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9f7755863d45bfb04a1498bd62e367)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:56.726988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:56.728267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:56.764534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1175/2362] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:51:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:51:57.020835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:57.023279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:57.037567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef049e-9f3e-4729-a3da-e696622e59fe service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:57.054757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:57.055491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:57.055702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:57.055980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] PATH_INFO: `/v3/users/4ce12f8401f3479ba14e5e1c3133a71d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:57.057647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce12f8401f3479ba14e5e1c3133a71d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:57.059897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:57.064371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9f7755863d45bfb04a1498bd62e367'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.065468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9f7755863d45bfb04a1498bd62e367'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.080775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9f7755863d45bfb04a1498bd62e367'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.345577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9f7755863d45bfb04a1498bd62e367'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.346283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9f7755863d45bfb04a1498bd62e367'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.369261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1188/2363] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:56 2022] DELETE /identity/v3/users/1d9f7755863d45bfb04a1498bd62e367 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:51:57.584237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce12f8401f3479ba14e5e1c3133a71d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.767964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce12f8401f3479ba14e5e1c3133a71d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.768937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce12f8401f3479ba14e5e1c3133a71d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.780677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:57.783956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:57.784208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce12f8401f3479ba14e5e1c3133a71d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.831389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce12f8401f3479ba14e5e1c3133a71d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:57.843144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1176/2364] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:51:57 2022] DELETE /identity/v3/users/4ce12f8401f3479ba14e5e1c3133a71d => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:57.874450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575fa6c-ce67-47ac-8559-30aa9fe9a2e8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:57.882782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:57.978174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:57.979245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:57.979718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:57.980147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] PATH_INFO: `/v3/projects/8af535d51fd345f8a9584cc6aaa38bda` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:57.981702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af535d51fd345f8a9584cc6aaa38bda)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:57.984094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:58.251539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af535d51fd345f8a9584cc6aaa38bda', 'request_id': 'req-010d1de7-5059-4b42-be9b-46ff26616571'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.259653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af535d51fd345f8a9584cc6aaa38bda', 'request_id': 'req-010d1de7-5059-4b42-be9b-46ff26616571'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.308113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:58.309806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:58.326297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1187c5-fc09-4c21-b656-44c84b2cd2d7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:58.407650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af535d51fd345f8a9584cc6aaa38bda', 'request_id': 'req-010d1de7-5059-4b42-be9b-46ff26616571'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.424309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:58.425129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.425588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.428248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] PATH_INFO: `/v3/projects/b71890b7a23944ee939c4bd13408063f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.428435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] The token cache is being invalidate because project 8af535d51fd345f8a9584cc6aaa38bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:58.428547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71890b7a23944ee939c4bd13408063f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:58.428668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.430328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-010d1de7-5059-4b42-be9b-46ff26616571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.430468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:58.433246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1189/2365] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:57 2022] DELETE /identity/v3/projects/8af535d51fd345f8a9584cc6aaa38bda => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:58.444842 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eabb5-aef4-4088-827b-dae0301d55b1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.448500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eabb5-aef4-4088-827b-dae0301d55b1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.448929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eabb5-aef4-4088-827b-dae0301d55b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.480520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a89eabb5-aef4-4088-827b-dae0301d55b1 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:58.481921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a89eabb5-aef4-4088-827b-dae0301d55b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:58.498452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1190/2366] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:58.513208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.513208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.513208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.532383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:58.533478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:58.546915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1191/2367] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:51:58.548148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:58.565385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8601507-54d4-4b2d-a88b-4bd49410afbf None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:58.580495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:58.581627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.581779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.582016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.582688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:58.583009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:58.585410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:58.605575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1192/2368] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:58.607063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:58.634672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d548-6971-476f-8fe5-3e152c76ffe7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:58.634672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:58.634672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.634672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.635201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.635201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:58.637153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:58.709475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:58.712474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71890b7a23944ee939c4bd13408063f', 'request_id': 'req-70f87669-2298-4ab2-997a-882c9f4ee530'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.834722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1193/2369] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:58 2022] POST /identity/v3/projects => generated 369 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:51:58.845635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:58.899046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71890b7a23944ee939c4bd13408063f', 'request_id': 'req-70f87669-2298-4ab2-997a-882c9f4ee530'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.909829 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71890b7a23944ee939c4bd13408063f', 'request_id': 'req-70f87669-2298-4ab2-997a-882c9f4ee530'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.931114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] The token cache is being invalidate because project b71890b7a23944ee939c4bd13408063f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:51:58.932112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.936933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70f87669-2298-4ab2-997a-882c9f4ee530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:51:58.950765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1177/2370] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:51:58 2022] DELETE /identity/v3/projects/b71890b7a23944ee939c4bd13408063f => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:58.954563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff7a2b-27dd-4933-81c7-574ba305cba3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:58.954563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff7a2b-27dd-4933-81c7-574ba305cba3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:58.954563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff7a2b-27dd-4933-81c7-574ba305cba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:58.981991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ffff7a2b-27dd-4933-81c7-574ba305cba3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:58.999935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ffff7a2b-27dd-4933-81c7-574ba305cba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:59.016157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a327055-3c1c-441b-b6c2-ff9140bc1804 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.023845 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.024501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.025269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.026577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1178/2371] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:59.028814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:51:59.032312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.032312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.032935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.034379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.034660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.034936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.068041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:59.069898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:59.084990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1179/2372] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:59.089069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.101471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7d65-8343-4b7b-b1d7-808d328f1c2d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.108222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.108934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.109367 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.109507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.110201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:51:59.122725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.125154 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.139183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1194/2373] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:58 2022] POST /identity/v3/users => generated 285 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:59.148239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.153104 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:59.168962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1180/2374] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:51:59.174906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.176378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25194-019d-48c5-a875-11e6ed7411c0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.183535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.184470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.184907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.186027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.186350 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.188203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.193018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47649c2b-403c-48b9-ad78-8aaa7e72b470 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.205117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.205775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.206082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.206359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.207017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.209139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.212489 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1195/2375] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:59.219444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.233190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bc680-561e-4aa2-ac66-9d07242c3db2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.239079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.239681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.240282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.240918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.241498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.243162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.256279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1196/2376] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:59.260798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.273059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6200cd-fe28-40ff-82ad-b3db88172b6d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.280666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fb772-97b1-44fa-8994-87ec8ee14b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.281765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fb772-97b1-44fa-8994-87ec8ee14b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.282279 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fb772-97b1-44fa-8994-87ec8ee14b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.282806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fb772-97b1-44fa-8994-87ec8ee14b79 admin admin] PATH_INFO: `/v3/projects/80235785da364f769b8020895b56013a/users/00757702b25b4ad397be00d0d59470d8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.312569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fb772-97b1-44fa-8994-87ec8ee14b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80235785da364f769b8020895b56013a, user_id=00757702b25b4ad397be00d0d59470d8, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.320891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fb772-97b1-44fa-8994-87ec8ee14b79 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.350923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1181/2377] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:59 2022] POST /identity/v3/projects => generated 403 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:59.358248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.452574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa343b2-9661-458a-bab1-3ae328daf7e0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.458649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.459471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.459895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.461059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.461436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.466132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.485569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1197/2378] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:59 2022] PUT /identity/v3/projects/80235785da364f769b8020895b56013a/users/00757702b25b4ad397be00d0d59470d8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:51:59.496671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.497043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.497381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.529783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] MFA Rules not processed for user `00757702b25b4ad397be00d0d59470d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:51:59.532247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:51:59.612180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1182/2379] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:59 2022] POST /identity/v3/users => generated 302 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:59.620606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.620878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1198/2380] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:51:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:51:59.621843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.635565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a9239-85f2-4e5a-a548-325066a0ee0a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.715706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0116a4-718e-4af3-bc6d-1dc3df7db78b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.719649 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.720559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.720953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.721315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.722396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.724242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.724503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.725980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.726261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.726737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.728271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.731395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.745882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1183/2381] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:59.749199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.760907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857d6d-f9d2-4733-9203-62c2938c0385 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.774639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.774639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.774639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.774639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.775063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.775063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.792617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1184/2382] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:51:59.793757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:51:59.807100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fee6d-5a65-4af5-a351-e8098c238c89 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:51:59.813225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa47e0-0302-44ff-b6ff-29c1556ccf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:51:59.814004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa47e0-0302-44ff-b6ff-29c1556ccf9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:51:59.814347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa47e0-0302-44ff-b6ff-29c1556ccf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:51:59.827450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa47e0-0302-44ff-b6ff-29c1556ccf9c admin admin] PATH_INFO: `/v3/projects/8b886d25ed8d44f39794df3a05d39856/users/8dfd60d600a54672b3ac756974a4fa91/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:51:59.833452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa47e0-0302-44ff-b6ff-29c1556ccf9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b886d25ed8d44f39794df3a05d39856, user_id=8dfd60d600a54672b3ac756974a4fa91, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:51:59.837881 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa47e0-0302-44ff-b6ff-29c1556ccf9c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:51:59.907537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1199/2383] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:51:59 2022] POST /identity/v3/projects => generated 369 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:51:59.912683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.021661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb74abb-d4a1-4649-9782-a17214bddeec admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:00.031297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:00.033295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.033295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.033295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.033295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:00.035301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:00.048387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1185/2384] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:51:59 2022] PUT /identity/v3/projects/8b886d25ed8d44f39794df3a05d39856/users/8dfd60d600a54672b3ac756974a4fa91/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:00.050926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.051380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.051751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.096852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:00.096852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:00.161579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1200/2385] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:51:59 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:00.167736 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.200379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcdee6-cd48-4406-8903-15ef0a57b9b6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:00.238701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1186/2386] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:00.249276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.251724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:00.305029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:00.305029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.305029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.305029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.305689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:00.308108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:00.326214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1201/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:00.330814 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.340475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ce603-f121-4465-b8d6-09a5fbd50916 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:00.345569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c69eb-7608-4262-9326-97eaf768f275 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:00.345823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:00.346864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.347258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.347699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.353834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:00.354975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.355935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.356540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] PATH_INFO: `/v3/projects/864b9b58a10047c58e7d57cc0eaf3145/users/54ad587def3e4df6bdfb8939dd89574c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.359736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:00.361795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:00.378285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1187/2388] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:00.385317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864b9b58a10047c58e7d57cc0eaf3145, user_id=54ad587def3e4df6bdfb8939dd89574c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:00.387573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:00.388620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:00.541608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1202/2389] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:00 2022] PUT /identity/v3/projects/864b9b58a10047c58e7d57cc0eaf3145/users/54ad587def3e4df6bdfb8939dd89574c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:00.548156 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.560804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf311869-b41d-437e-a50d-e249e84f86fc service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:00.676041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:00.692424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec78ce8-bb04-4836-8c6e-869532cd77c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:00.694447 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec78ce8-bb04-4836-8c6e-869532cd77c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.695243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec78ce8-bb04-4836-8c6e-869532cd77c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.695723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec78ce8-bb04-4836-8c6e-869532cd77c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.697117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec78ce8-bb04-4836-8c6e-869532cd77c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:00.701069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec78ce8-bb04-4836-8c6e-869532cd77c7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:00.722815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1188/2390] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:00.727633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.743517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470a3b-4d1b-438a-a772-1d0d68250d90 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:00.767168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5ef4e-ac57-454a-98eb-ca151b590440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:00.767912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5ef4e-ac57-454a-98eb-ca151b590440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.768193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5ef4e-ac57-454a-98eb-ca151b590440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.768511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5ef4e-ac57-454a-98eb-ca151b590440 admin admin] PATH_INFO: `/v3/projects/864b9b58a10047c58e7d57cc0eaf3145/users/54ad587def3e4df6bdfb8939dd89574c/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.770639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5ef4e-ac57-454a-98eb-ca151b590440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=864b9b58a10047c58e7d57cc0eaf3145, user_id=54ad587def3e4df6bdfb8939dd89574c, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:00.774993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5ef4e-ac57-454a-98eb-ca151b590440 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:00.791230 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1203/2391] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:00 2022] PUT /identity/v3/projects/864b9b58a10047c58e7d57cc0eaf3145/users/54ad587def3e4df6bdfb8939dd89574c/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:00.796084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.796621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.796712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.811696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:00.812047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:00.812474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:00.831108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:00.831108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:00.870881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] MFA Rules not processed for user `54ad587def3e4df6bdfb8939dd89574c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:00.872989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:00.962697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1189/2392] 10.222.0.17 () {58 vars in 967 bytes} [Wed Aug 24 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:00.982783 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:00.986161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:01.006266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1204/2393] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:01.023375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:01.026930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:01.154983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd31f16-d31d-4675-90fa-97914b6a9c8d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:01.163441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:01.164418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:01.164566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:01.164905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:01.170641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b60b3-602a-42aa-8258-f22af98d9a01 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:01.179950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5b529-0f12-45c3-a3b9-5bcd22287abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:01.180142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5b529-0f12-45c3-a3b9-5bcd22287abd service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:01.180711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5b529-0f12-45c3-a3b9-5bcd22287abd service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:01.181111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5b529-0f12-45c3-a3b9-5bcd22287abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:01.183043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:01.186831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:01.201394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5b529-0f12-45c3-a3b9-5bcd22287abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:01.205084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5b529-0f12-45c3-a3b9-5bcd22287abd service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:01.215286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1190/2394] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:52:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:52:01.232882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1205/2395] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:01.581026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:01.581026 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:01.590399 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309ecbc-abcd-421c-bc9d-4ad79f2d73aa service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:01.605096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:01.605981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:01.606344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:01.606696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:01.708084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:01.710448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:01.725353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:01.725610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1191/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:02.305295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:03.361246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:03.363752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:03.363752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:03.398224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] MFA Rules not processed for user `8dfd60d600a54672b3ac756974a4fa91`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:03.399559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:03.503503 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1206/2397] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:03.508444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:03.597181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea29a2-b966-4bce-896f-a5f728c6c426 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:03.602676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:03.603315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:03.603777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:03.604045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:03.604707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:03.606641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:03.876722 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1192/2398] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:03 2022] POST /identity/v3/projects => generated 403 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:03.882652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:03.908810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bc4fa-05ae-40b2-8914-2adf42d56f1c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:03.981644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:03.982508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:03.982846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:03.983213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:03.983977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:03.986311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:04.172094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1207/2399] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:03 2022] POST /identity/v3/users => generated 308 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:04.176596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:04.187201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932ac6e-a3f1-4d50-a6a5-95c54773e56a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:04.191907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:04.192552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:04.192827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:04.193096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:04.194281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:04.196042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:04.207404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1193/2400] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:04.215437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:04.226827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:04.236299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b55c2-ea96-48d4-89a5-cdc6c4018345 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:04.245421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:04.245421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:04.245421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:04.245827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] PATH_INFO: `/v3/projects/02af537ef76e416296a562ec399f5443/users/da18d7b3125b44fd9946eef3b3e7dfdf/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:04.278322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02af537ef76e416296a562ec399f5443, user_id=da18d7b3125b44fd9946eef3b3e7dfdf, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:04.292715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:04.497015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1208/2401] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:04 2022] PUT /identity/v3/projects/02af537ef76e416296a562ec399f5443/users/da18d7b3125b44fd9946eef3b3e7dfdf/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:04.500957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:04.513297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727eaae-ee7b-4389-8520-6ecaf86532f6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:04.601547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:04.602188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:04.602469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:04.602741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:04.603406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:04.605169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:04.617974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1194/2402] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:04.623986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:04.636866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fcaed-46e9-4c5c-817e-0b09fe5708b2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:04.645512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0bdc9-c1aa-49e1-abfe-39b33de9d887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:04.646908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0bdc9-c1aa-49e1-abfe-39b33de9d887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:04.647271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0bdc9-c1aa-49e1-abfe-39b33de9d887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:04.647632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0bdc9-c1aa-49e1-abfe-39b33de9d887 admin admin] PATH_INFO: `/v3/projects/02af537ef76e416296a562ec399f5443/users/da18d7b3125b44fd9946eef3b3e7dfdf/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:04.649908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0bdc9-c1aa-49e1-abfe-39b33de9d887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02af537ef76e416296a562ec399f5443, user_id=da18d7b3125b44fd9946eef3b3e7dfdf, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:04.653286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0bdc9-c1aa-49e1-abfe-39b33de9d887 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:04.668941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1209/2403] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:04 2022] PUT /identity/v3/projects/02af537ef76e416296a562ec399f5443/users/da18d7b3125b44fd9946eef3b3e7dfdf/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:04.734230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:04.737569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:04.758745 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b552b-0082-4b7f-b036-3c6b50075aea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:04.838948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:04.839669 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:04.840110 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:04.840599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] PATH_INFO: `/v3/users/b94aa32205404578afd23f6bc4f8c14d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:04.844756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94aa32205404578afd23f6bc4f8c14d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:04.844756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:04.848956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:05.115265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94aa32205404578afd23f6bc4f8c14d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:05.210394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94aa32205404578afd23f6bc4f8c14d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:05.210394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94aa32205404578afd23f6bc4f8c14d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:05.217187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94aa32205404578afd23f6bc4f8c14d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:05.241251 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94aa32205404578afd23f6bc4f8c14d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:05.249224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1195/2404] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:04 2022] DELETE /identity/v3/users/b94aa32205404578afd23f6bc4f8c14d => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:05.254624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae8675-807a-4be7-8cf6-c3ed19b5691d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:05.254978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae8675-807a-4be7-8cf6-c3ed19b5691d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:05.255363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae8675-807a-4be7-8cf6-c3ed19b5691d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:05.290361 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-20ae8675-807a-4be7-8cf6-c3ed19b5691d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:05.291900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-20ae8675-807a-4be7-8cf6-c3ed19b5691d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:05.405383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1210/2405] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:05.420395 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:05.422162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:05.545680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a01e8-5732-4931-84fb-359d14576c33 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:05.566493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:05.567309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:05.567593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:05.567876 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:05.607019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:05.619109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:05.621675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:05.643992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1196/2406] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:06.576498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e51485-2681-40c5-a444-75506a899b66 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:06.577171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e51485-2681-40c5-a444-75506a899b66 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:06.585015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e51485-2681-40c5-a444-75506a899b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:06.642157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-95e51485-2681-40c5-a444-75506a899b66 None None] MFA Rules not processed for user `da18d7b3125b44fd9946eef3b3e7dfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:06.646184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-95e51485-2681-40c5-a444-75506a899b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:06.786406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:06.788711 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:06.805944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48271587-22d6-42df-bced-8547003c2ef9 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:06.811239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:06.811889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:06.812166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:06.812479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] PATH_INFO: `/v3/projects/758d557cade148a291927dbdda413ad6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:06.814123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=758d557cade148a291927dbdda413ad6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:06.817030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:06.817958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1211/2407] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:06 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:06.984567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758d557cade148a291927dbdda413ad6', 'request_id': 'req-0c05353f-9c67-41c7-9992-d9e3f68982a7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:07.101911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758d557cade148a291927dbdda413ad6', 'request_id': 'req-0c05353f-9c67-41c7-9992-d9e3f68982a7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:07.114625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758d557cade148a291927dbdda413ad6', 'request_id': 'req-0c05353f-9c67-41c7-9992-d9e3f68982a7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:07.137696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] The token cache is being invalidate because project 758d557cade148a291927dbdda413ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:07.138330 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:07.139427 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c05353f-9c67-41c7-9992-d9e3f68982a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:07.142227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1197/2408] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:06 2022] DELETE /identity/v3/projects/758d557cade148a291927dbdda413ad6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:07.152674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:07.159387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:07.161062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:07.207109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:07.210526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:07.304534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:07.327962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1212/2409] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:07.337227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:07.337852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:07.338288 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:07.385037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:07.388666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:07.408543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1198/2410] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:07.409291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:07.427006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a3afc-8aa5-43be-b0fa-5c29ae68d411 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:07.434375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:07.435552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:07.436133 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:07.436812 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:07.438067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:07.438681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:07.441805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:07.463124 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1213/2411] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:52:07.467272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:07.481239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d277ab-4ee7-4f7e-850b-9bc5cfdfaa1e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:07.503834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:07.504132 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:07.504502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:07.504869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:07.505612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:07.508664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:07.616536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1199/2412] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:07 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:07.622567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:07.639762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeae43e-f453-46b4-813b-156c5bae99a5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:07.786264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:07.786264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:07.786264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:07.786264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:07.787956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:07.795423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:07.962968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1214/2413] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:07 2022] POST /identity/v3/users => generated 287 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:07.965304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:07.982980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b86e65-98b4-4c6c-8fc6-059d30722e15 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:08.001081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:08.003356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.004557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.005435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.008071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:08.013094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:08.033787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1200/2414] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:08.045363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:08.064009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1397a-3f88-451c-851e-997a43939a89 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:08.073077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a18c4b-124e-427d-ad1b-70b13945943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:08.074422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18c4b-124e-427d-ad1b-70b13945943e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.075038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18c4b-124e-427d-ad1b-70b13945943e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.075893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a18c4b-124e-427d-ad1b-70b13945943e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.077630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a18c4b-124e-427d-ad1b-70b13945943e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:08.079620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a18c4b-124e-427d-ad1b-70b13945943e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:08.105497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1215/2415] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:08.110229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:08.124718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955976ae-f7f5-4a7f-95f7-861df51ab8df admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:08.154760 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:08.155446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.155723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.156006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] PATH_INFO: `/v3/projects/f0f36a9b4a1b4a2ead9c88fc5db02f47/users/a62a3e2f0f6042be9ffd0a2679bf5258/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.182078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f36a9b4a1b4a2ead9c88fc5db02f47, user_id=a62a3e2f0f6042be9ffd0a2679bf5258, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:08.186140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:08.288378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1201/2416] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:08 2022] PUT /identity/v3/projects/f0f36a9b4a1b4a2ead9c88fc5db02f47/users/a62a3e2f0f6042be9ffd0a2679bf5258/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:08.357090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.357090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.357090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.407030 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] MFA Rules not processed for user `a62a3e2f0f6042be9ffd0a2679bf5258`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:08.409393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:08.444396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:08.448074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:08.468708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:08.578304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1216/2417] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:08.593304 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:08.594872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:08.603486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bb6f8-1e44-4072-9976-2e0b1376d2c4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:08.611119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:08.611895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.612253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.612771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] PATH_INFO: `/v3/users/00757702b25b4ad397be00d0d59470d8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.614306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00757702b25b4ad397be00d0d59470d8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:08.616861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:08.678796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:08.678796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db15fc4-6da7-4e82-b631-c277b5ef83e3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:08.678796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:08.678796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.679605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.679605 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.687790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:08.689280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:08.700607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1217/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:08.854055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:08.855530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:08.877082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7559fc-7790-44ae-99ea-799e314261aa service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:08.887673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:08.888825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:08.889226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:08.889816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] PATH_INFO: `/v3/users/5f10bf98bc784be7bf2f8fde1ddbef44` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:08.891452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f10bf98bc784be7bf2f8fde1ddbef44)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:08.893656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:09.107584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00757702b25b4ad397be00d0d59470d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.291882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00757702b25b4ad397be00d0d59470d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.297118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00757702b25b4ad397be00d0d59470d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.302619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00757702b25b4ad397be00d0d59470d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.315648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00757702b25b4ad397be00d0d59470d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.324854 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1202/2419] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:08 2022] DELETE /identity/v3/users/00757702b25b4ad397be00d0d59470d8 => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:09.326356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:09.328828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:09.413609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ddfc2-bf11-4b39-b70f-f76a6398aa9e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:09.420946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:09.422522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:09.422522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:09.422522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] PATH_INFO: `/v3/users/8dfd60d600a54672b3ac756974a4fa91` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:09.429192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfd60d600a54672b3ac756974a4fa91)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:09.434518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:09.434727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f10bf98bc784be7bf2f8fde1ddbef44'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.434727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f10bf98bc784be7bf2f8fde1ddbef44'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.448721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f10bf98bc784be7bf2f8fde1ddbef44'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.582288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f10bf98bc784be7bf2f8fde1ddbef44'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.583017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-42fc7a73-e3ba-441f-aafd-9394e09b4201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f10bf98bc784be7bf2f8fde1ddbef44'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:09.608374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1218/2420] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:08 2022] DELETE /identity/v3/users/5f10bf98bc784be7bf2f8fde1ddbef44 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:09.610623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:09.611122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:09.611621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:09.641235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:09.642472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:09.716366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1219/2421] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:09.730508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:09.732823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:09.829718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94195e32-3a95-4cfa-b769-d7908c36525f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:09.836059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:09.837123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:09.837403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:09.837688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:09.852412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:09.854543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:09.866803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1220/2422] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:09.876562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfd60d600a54672b3ac756974a4fa91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.024675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:10.026097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:10.045610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfd60d600a54672b3ac756974a4fa91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.046569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfd60d600a54672b3ac756974a4fa91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.072505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd7c69-d432-48ad-be0f-790b989d46c4 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:10.074484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfd60d600a54672b3ac756974a4fa91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.083115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:10.084949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:10.085687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:10.086579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] PATH_INFO: `/v3/projects/05a8a1a0e0334559ab3fc9a63685fa69` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:10.088393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a8a1a0e0334559ab3fc9a63685fa69)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:10.091515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:10.092560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfd60d600a54672b3ac756974a4fa91'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.100792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1203/2423] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:09 2022] DELETE /identity/v3/users/8dfd60d600a54672b3ac756974a4fa91 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:10.349093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a8a1a0e0334559ab3fc9a63685fa69', 'request_id': 'req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.363918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a8a1a0e0334559ab3fc9a63685fa69', 'request_id': 'req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.561201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a8a1a0e0334559ab3fc9a63685fa69', 'request_id': 'req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.574889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:10.576391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:10.577286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] The token cache is being invalidate because project 05a8a1a0e0334559ab3fc9a63685fa69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:10.577662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.578475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:10.587258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1221/2424] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:10 2022] DELETE /identity/v3/projects/05a8a1a0e0334559ab3fc9a63685fa69 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:10.588611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:10.590244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:10.671490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3862529-6543-466a-a62f-cc41f577504a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:10.676555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:10.677221 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:10.677498 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:10.678446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] PATH_INFO: `/v3/projects/80235785da364f769b8020895b56013a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:10.679568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=80235785da364f769b8020895b56013a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:10.681567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:10.682468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25d8b4-7e58-43cd-9111-e5a1eb2dac67 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:10.687798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:10.688474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:10.688758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:10.689042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] PATH_INFO: `/v3/projects/8b886d25ed8d44f39794df3a05d39856` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:10.690250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b886d25ed8d44f39794df3a05d39856)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:10.692182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:11.013685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b886d25ed8d44f39794df3a05d39856', 'request_id': 'req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.014916 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80235785da364f769b8020895b56013a', 'request_id': 'req-debe3860-bb3c-4763-8ce3-ccc1daa740ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.020517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b886d25ed8d44f39794df3a05d39856', 'request_id': 'req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.182224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80235785da364f769b8020895b56013a', 'request_id': 'req-debe3860-bb3c-4763-8ce3-ccc1daa740ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.182996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:11.189836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80235785da364f769b8020895b56013a', 'request_id': 'req-debe3860-bb3c-4763-8ce3-ccc1daa740ed'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.201331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] The token cache is being invalidate because project 80235785da364f769b8020895b56013a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:11.201622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.202396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.204012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1204/2425] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:10 2022] DELETE /identity/v3/projects/80235785da364f769b8020895b56013a => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:11.205813 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:11.206227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:11.277379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe3860-bb3c-4763-8ce3-ccc1daa740ed admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:11.338329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:11.339021 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:11.339300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:11.339571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] PATH_INFO: `/v3/users/0f449d61d6b141feb0cf65f4f89f35d1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:11.340776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f449d61d6b141feb0cf65f4f89f35d1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:11.342744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:11.405922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b886d25ed8d44f39794df3a05d39856', 'request_id': 'req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.418628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] The token cache is being invalidate because project 8b886d25ed8d44f39794df3a05d39856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:11.419042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.441053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.441053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1222/2426] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:10 2022] DELETE /identity/v3/projects/8b886d25ed8d44f39794df3a05d39856 => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:11.441053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:11.441053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:11.441053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53df82-2d0f-4d1f-ba90-fa39d9e6cadc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:11.442658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:11.443429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:11.443812 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:11.444092 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] PATH_INFO: `/v3/users/54ad587def3e4df6bdfb8939dd89574c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:11.445321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ad587def3e4df6bdfb8939dd89574c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:11.447253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:11.734497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:11.752109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f449d61d6b141feb0cf65f4f89f35d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.884588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ad587def3e4df6bdfb8939dd89574c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.885321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ad587def3e4df6bdfb8939dd89574c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.894187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ad587def3e4df6bdfb8939dd89574c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.928961 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f449d61d6b141feb0cf65f4f89f35d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.929769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f449d61d6b141feb0cf65f4f89f35d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.937600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f449d61d6b141feb0cf65f4f89f35d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.954380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f449d61d6b141feb0cf65f4f89f35d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:11.961503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1205/2427] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:11 2022] DELETE /identity/v3/users/0f449d61d6b141feb0cf65f4f89f35d1 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:11.962303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:11.963694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:12.038102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412a6f0-4543-4a95-bacb-bdb94892b702 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:12.043552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:12.044405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:12.044769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:12.045052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] PATH_INFO: `/v3/users/da18d7b3125b44fd9946eef3b3e7dfdf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:12.046255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da18d7b3125b44fd9946eef3b3e7dfdf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:12.048555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:12.165227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ad587def3e4df6bdfb8939dd89574c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.166275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ad587def3e4df6bdfb8939dd89574c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.186216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1223/2428] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:11 2022] DELETE /identity/v3/users/54ad587def3e4df6bdfb8939dd89574c => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:12.290012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:12.291918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:12.346277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da18d7b3125b44fd9946eef3b3e7dfdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.378920 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77b369-b709-418e-bead-377d4a853800 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:12.386664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:12.387312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:12.394105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:12.394105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] PATH_INFO: `/v3/projects/fa06923c74854d6c82a342edd8a670af` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:12.394105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa06923c74854d6c82a342edd8a670af)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:12.394105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:12.467991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da18d7b3125b44fd9946eef3b3e7dfdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.468596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da18d7b3125b44fd9946eef3b3e7dfdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.477596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da18d7b3125b44fd9946eef3b3e7dfdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.492052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7a34b596-84d4-4639-ae6d-57b0af413208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da18d7b3125b44fd9946eef3b3e7dfdf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.500476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1206/2429] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:11 2022] DELETE /identity/v3/users/da18d7b3125b44fd9946eef3b3e7dfdf => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:12.538602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa06923c74854d6c82a342edd8a670af', 'request_id': 'req-270adffb-3513-4be0-9316-867c5e6bbcae'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.548058 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa06923c74854d6c82a342edd8a670af', 'request_id': 'req-270adffb-3513-4be0-9316-867c5e6bbcae'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.670251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa06923c74854d6c82a342edd8a670af', 'request_id': 'req-270adffb-3513-4be0-9316-867c5e6bbcae'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.688161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] The token cache is being invalidate because project fa06923c74854d6c82a342edd8a670af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:12.688481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.689182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-270adffb-3513-4be0-9316-867c5e6bbcae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:12.690910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1224/2430] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:12 2022] DELETE /identity/v3/projects/fa06923c74854d6c82a342edd8a670af => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:12.698688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:12.699078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:12.699359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:12.727088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:12.731591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:12.811054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1207/2431] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:12.822586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:12.822586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:12.824134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:12.845682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:12.847437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:12.857878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1225/2432] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:12.868049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:12.880196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0a521-e717-4e21-8d28-7884f1a5c3f4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:12.887532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:12.888283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:12.888698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:12.889039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:12.889871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:12.890241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:12.893988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:12.918679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1208/2433] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:12.919764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:12.932445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf652b-e419-4b13-a980-d2fe070846e8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:12.938584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:12.939401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:12.939749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:12.941355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:12.942318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:12.944774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.021827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.028701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.040227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7110f-a3a2-469d-a75b-3357d1a08864 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.046485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.047137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.047489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.047852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] PATH_INFO: `/v3/projects/02af537ef76e416296a562ec399f5443` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.049437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02af537ef76e416296a562ec399f5443)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.053664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.114313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1226/2434] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:12 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:13.121988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.124009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.197873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9ab04-61a0-48b6-91e4-d38e4489a351 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.203251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.204239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.204557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.204841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] PATH_INFO: `/v3/projects/864b9b58a10047c58e7d57cc0eaf3145` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.206562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=864b9b58a10047c58e7d57cc0eaf3145)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.210891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.335645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02af537ef76e416296a562ec399f5443', 'request_id': 'req-2c2af35e-f75b-4b25-a5e4-397de2080b03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.374025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '864b9b58a10047c58e7d57cc0eaf3145', 'request_id': 'req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.380672 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '864b9b58a10047c58e7d57cc0eaf3145', 'request_id': 'req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.459670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02af537ef76e416296a562ec399f5443', 'request_id': 'req-2c2af35e-f75b-4b25-a5e4-397de2080b03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.467273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02af537ef76e416296a562ec399f5443', 'request_id': 'req-2c2af35e-f75b-4b25-a5e4-397de2080b03'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.480931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] The token cache is being invalidate because project 02af537ef76e416296a562ec399f5443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:13.481225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.481964 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.483571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1209/2435] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:13 2022] DELETE /identity/v3/projects/02af537ef76e416296a562ec399f5443 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:13.484620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.486577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.560208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2af35e-f75b-4b25-a5e4-397de2080b03 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.573753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '864b9b58a10047c58e7d57cc0eaf3145', 'request_id': 'req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.587703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] The token cache is being invalidate because project 864b9b58a10047c58e7d57cc0eaf3145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:13.587913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.588584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3a39b8e1-f85d-4cb6-a439-813ba16ac0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:13.590322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1227/2436] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:13 2022] DELETE /identity/v3/projects/864b9b58a10047c58e7d57cc0eaf3145 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:13.592024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87edb26-53b6-451d-a501-a43d07246147 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.592024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87edb26-53b6-451d-a501-a43d07246147 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.592248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87edb26-53b6-451d-a501-a43d07246147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.614734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c87edb26-53b6-451d-a501-a43d07246147 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:13.616630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c87edb26-53b6-451d-a501-a43d07246147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.623618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.624243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.624554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.624830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.625462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.627522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.690603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1228/2437] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:13.691862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa06a3-3379-45a2-81d5-d9d29ef351ac None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.692141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa06a3-3379-45a2-81d5-d9d29ef351ac None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.692441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa06a3-3379-45a2-81d5-d9d29ef351ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.714443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-70aa06a3-3379-45a2-81d5-d9d29ef351ac None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:13.715655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-70aa06a3-3379-45a2-81d5-d9d29ef351ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.723649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1229/2438] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:13.725662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b982c-95ad-4e44-9ba6-6c84ad8cea0c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.725818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b982c-95ad-4e44-9ba6-6c84ad8cea0c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.726144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b982c-95ad-4e44-9ba6-6c84ad8cea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.746684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-616b982c-95ad-4e44-9ba6-6c84ad8cea0c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:13.747985 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-616b982c-95ad-4e44-9ba6-6c84ad8cea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.760069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1230/2439] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:13.760069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.760069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.760069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.778006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:13.779217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.787257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1231/2440] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:13.789115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.804085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499ba94-1074-42b0-a675-639b66f89fd9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.810138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1210/2441] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/users => generated 289 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:13.811594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.813816 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.814555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.817326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.817326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.817326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:13.817326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.817980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.824061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6da1a-063b-408a-b395-4ebeda1b4fff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.830468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.831025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1232/2442] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:13.831140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.831346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.831718 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.831968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.832276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:13.832609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.834401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.835967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:13.848252 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1211/2443] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:13.850114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.855641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c7295-935a-4e7e-b58a-fb3d0d62df0b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.861076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.861755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.862041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.862309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.862639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbcc40-3e31-4d9b-9d3b-bdfd8dd778f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.863001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.864703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.869248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.869861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.870139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.870411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.871051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.873090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:13.877088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1233/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:13.878140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:13.888795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c96e4-c625-4344-b1e4-d8b56fd601a0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:13.893353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:13.893964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:13.894382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:13.894588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:13.895291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:13.897452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.044825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1212/2445] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:14.045942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.084492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1234/2446] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:13 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:14.089107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.116810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4294-3d67-4c08-892e-cc38d59be759 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.167151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:14.171870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3649b-0de0-46d9-b879-414709c242aa admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.179784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.180916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.181156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.181611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.182352 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.184431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.187060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.187753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.188035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.188302 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.189048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.190591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.201108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1213/2447] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:14.202060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.214905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f50db-9b28-402d-b985-78a5bc8e43aa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.222804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.223662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.224184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.224684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.226609 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.230378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.363391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1235/2448] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:14 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:14.364410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.375407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac74ac6-c921-4af7-9ada-87c3ec9c435a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.382085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.382770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.383054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.383381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] PATH_INFO: `/v3/projects/f18db8d724d4436cb1721859066ac74d/users/aaa4ddbee1034946a1f4dfe751a2b5cd/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.387959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1214/2449] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:14 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:14.388830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.399407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566c438-bc68-4854-8bfb-2eb2403d75cc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.399837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18db8d724d4436cb1721859066ac74d, user_id=aaa4ddbee1034946a1f4dfe751a2b5cd, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.402912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.404524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.405125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.405402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.405667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.406334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.408196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.420407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1215/2450] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:14.421273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.431468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef01e7-510f-4d39-b2f4-b4c8203c2ce9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.436190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.436812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.437103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.437360 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.437988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.439592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.450072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1216/2451] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:14.450892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.461627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c8e6d-48a4-4af2-b37e-b6bde4614f45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.466851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.470366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.470366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.470366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.470366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.470366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.481998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1217/2452] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:14.482952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.493054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357212f4-545d-443f-aa48-1686aebe8d86 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.497676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.498260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.498535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.498804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.499438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.501094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.511689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1218/2453] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:14.512531 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.523838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210440e9-b010-4923-97bd-3489c8896878 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.528921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087fa1c-8251-41ff-9b27-9a902a35ab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.530064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fa1c-8251-41ff-9b27-9a902a35ab33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.530064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fa1c-8251-41ff-9b27-9a902a35ab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.530333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087fa1c-8251-41ff-9b27-9a902a35ab33 admin admin] PATH_INFO: `/v3/projects/f30a5fa4bd204a2ea790b9afa6f4e663/users/4cf154372a7b4d489e93908b681faf69/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.546953 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087fa1c-8251-41ff-9b27-9a902a35ab33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30a5fa4bd204a2ea790b9afa6f4e663, user_id=4cf154372a7b4d489e93908b681faf69, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.549700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087fa1c-8251-41ff-9b27-9a902a35ab33 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.595146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1236/2454] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:14 2022] PUT /identity/v3/projects/f18db8d724d4436cb1721859066ac74d/users/aaa4ddbee1034946a1f4dfe751a2b5cd/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:14.595703 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.711996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e1eec-b652-4bd1-836a-4014adf6feba admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.718130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1219/2455] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:14 2022] PUT /identity/v3/projects/f30a5fa4bd204a2ea790b9afa6f4e663/users/4cf154372a7b4d489e93908b681faf69/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:14.719624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.719969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.720305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.761616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:14.762844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:14.783378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaa675-38f2-4d8b-a1c1-046f974b00e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.784048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaa675-38f2-4d8b-a1c1-046f974b00e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.784333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaa675-38f2-4d8b-a1c1-046f974b00e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.784634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaa675-38f2-4d8b-a1c1-046f974b00e3 admin admin] PATH_INFO: `/v3/projects/4ed4a384fa1f42d1b644ea9cfb291b60/users/f2bcf5e4247a4e45a7fb2c0ba3b57d65/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.801054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaa675-38f2-4d8b-a1c1-046f974b00e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed4a384fa1f42d1b644ea9cfb291b60, user_id=f2bcf5e4247a4e45a7fb2c0ba3b57d65, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.804600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaa675-38f2-4d8b-a1c1-046f974b00e3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.828460 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1220/2456] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:14.829476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:14.830793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:14.844541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba4b0d-4ac6-4a9d-90c6-0a90a151443a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:14.849057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:14.855216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:14.855216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.855216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.855216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] PATH_INFO: `/v3/users/a62a3e2f0f6042be9ffd0a2679bf5258` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:14.855610 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62a3e2f0f6042be9ffd0a2679bf5258)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:14.857380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:14.987734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1237/2457] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:14 2022] PUT /identity/v3/projects/4ed4a384fa1f42d1b644ea9cfb291b60/users/f2bcf5e4247a4e45a7fb2c0ba3b57d65/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:14.989667 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:14.990090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:14.991023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.028296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] MFA Rules not processed for user `4cf154372a7b4d489e93908b681faf69`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:15.029679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.097530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1238/2458] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:15.098543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.099949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.183869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffb49c-f466-4626-bbc3-e61c74e9106e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.187876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2776-8635-431d-9adb-e7cf1d9dbb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.188531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2776-8635-431d-9adb-e7cf1d9dbb96 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.199341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2776-8635-431d-9adb-e7cf1d9dbb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.199341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2776-8635-431d-9adb-e7cf1d9dbb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.259329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa2776-8635-431d-9adb-e7cf1d9dbb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.260879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa2776-8635-431d-9adb-e7cf1d9dbb96 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:15.270615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1239/2459] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:15.272097 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.272401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.272674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.303591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] MFA Rules not processed for user `f2bcf5e4247a4e45a7fb2c0ba3b57d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:15.304204 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62a3e2f0f6042be9ffd0a2679bf5258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:15.304957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.376094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1240/2460] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:15.376945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.387519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1895cd-154d-4d0c-9b1a-c103a7e1599c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.392135 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b8955-9dbd-425c-97de-5a6ff3ddbdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.392769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8955-9dbd-425c-97de-5a6ff3ddbdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.393047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8955-9dbd-425c-97de-5a6ff3ddbdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.393311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b8955-9dbd-425c-97de-5a6ff3ddbdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.393921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b8955-9dbd-425c-97de-5a6ff3ddbdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.395840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b8955-9dbd-425c-97de-5a6ff3ddbdc0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:15.436551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62a3e2f0f6042be9ffd0a2679bf5258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:15.437335 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62a3e2f0f6042be9ffd0a2679bf5258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:15.446875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62a3e2f0f6042be9ffd0a2679bf5258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:15.462539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62a3e2f0f6042be9ffd0a2679bf5258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:15.469956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1221/2461] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:14 2022] DELETE /identity/v3/users/a62a3e2f0f6042be9ffd0a2679bf5258 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:15.471096 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.475306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.520113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1241/2462] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:15 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:15.520315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.520823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.521055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.535497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16a97d-e2bb-40e9-92f0-d40c31025406 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.540608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:15.542087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.588450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.589073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.589343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.589645 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.591455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.607015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1242/2463] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:15.607645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.612034 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.621616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a594f0-c60b-478e-b8ab-3e00b96456c5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.626239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.626837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.627110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.627378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.627994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.630003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:15.656643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.658161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:15.668920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1222/2464] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:15.669788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.671065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:15.740260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364358-a76e-40ed-b447-41d9662cced6 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.744574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.745824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.745824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.746031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.756334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.757927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:15.768075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1223/2465] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:15.796969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1243/2466] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:15 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:15.801079 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.812554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288c625-eb6b-4770-9581-6461c7ffb8f4 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.821756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.822810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.823303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.823781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.825421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.831089 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:15.838879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1224/2467] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:52:15.843166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:15.854215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be352833-c8ea-4bfb-98bd-63b154f0e7ec admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:15.860209 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:15.860449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:15.860723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:15.860990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] PATH_INFO: `/v3/projects/22ef9ab19bca4838b582fe42df6af338/users/288659d4cdef47948be0e4cf397d5cef/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:15.878408 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ef9ab19bca4838b582fe42df6af338, user_id=288659d4cdef47948be0e4cf397d5cef, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:15.897796 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:16.068391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1244/2468] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:15 2022] PUT /identity/v3/projects/22ef9ab19bca4838b582fe42df6af338/users/288659d4cdef47948be0e4cf397d5cef/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:16.072370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:16.086294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06de655-4e71-4158-91ad-29482b812284 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:16.154465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dcdc2-d271-4090-8923-897717fea9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:16.155842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dcdc2-d271-4090-8923-897717fea9bc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.155842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dcdc2-d271-4090-8923-897717fea9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.155842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dcdc2-d271-4090-8923-897717fea9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.156594 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dcdc2-d271-4090-8923-897717fea9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:16.158341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dcdc2-d271-4090-8923-897717fea9bc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:16.168872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1225/2469] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:16.173174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:16.184136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1df3f-cf0f-4aba-99cc-8c62010d11f3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:16.189447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:16.190114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.190391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.190669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] PATH_INFO: `/v3/projects/22ef9ab19bca4838b582fe42df6af338/users/288659d4cdef47948be0e4cf397d5cef/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.192691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ef9ab19bca4838b582fe42df6af338, user_id=288659d4cdef47948be0e4cf397d5cef, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:16.195900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:16.207421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1245/2470] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:16 2022] PUT /identity/v3/projects/22ef9ab19bca4838b582fe42df6af338/users/288659d4cdef47948be0e4cf397d5cef/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:16.223930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.224196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.224502 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.257946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] MFA Rules not processed for user `288659d4cdef47948be0e4cf397d5cef`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:16.259356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:16.334609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:16.337272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1226/2471] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:16.337640 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:16.360227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:16.361546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:16.425043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6c11-708e-4795-a31e-526407373fcc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:16.429827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20f5d2-bedf-480f-8399-c953957e7757 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:16.431762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:16.432738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.433227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.433497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce7a5-a150-4459-b64f-88db73730786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:16.433964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] PATH_INFO: `/v3/projects/f0f36a9b4a1b4a2ead9c88fc5db02f47` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.434097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce7a5-a150-4459-b64f-88db73730786 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.434362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce7a5-a150-4459-b64f-88db73730786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.434621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce7a5-a150-4459-b64f-88db73730786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.436318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f36a9b4a1b4a2ead9c88fc5db02f47)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:16.438496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:16.446317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ce7a5-a150-4459-b64f-88db73730786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:16.448013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ce7a5-a150-4459-b64f-88db73730786 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:16.460640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1227/2472] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:16.586594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f36a9b4a1b4a2ead9c88fc5db02f47', 'request_id': 'req-b4dd379e-ab44-4147-a971-61ab1941a9cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:16.593623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f36a9b4a1b4a2ead9c88fc5db02f47', 'request_id': 'req-b4dd379e-ab44-4147-a971-61ab1941a9cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:16.706967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f36a9b4a1b4a2ead9c88fc5db02f47', 'request_id': 'req-b4dd379e-ab44-4147-a971-61ab1941a9cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:16.721149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] The token cache is being invalidate because project f0f36a9b4a1b4a2ead9c88fc5db02f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:16.721448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:16.722201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b4dd379e-ab44-4147-a971-61ab1941a9cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:16.724683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1246/2473] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:16 2022] DELETE /identity/v3/projects/f0f36a9b4a1b4a2ead9c88fc5db02f47 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:16.730199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.730666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.731068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.793283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:16.794514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:16.879426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1228/2474] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:16.880027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.880027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.880027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.908100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:16.909506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:16.923983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1247/2475] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:16.929061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:16.939424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccbbde-403c-4d1e-90b3-50fbe4f7028c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:16.944623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:16.945253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:16.945526 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:16.946030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:16.946662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:16.946969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:16.948779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:16.963305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1229/2476] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:16.968178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:16.986406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:16.997227 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462117ba-09fc-432f-8354-db9387a74b5a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:17.004626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:17.006889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.007768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.008464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.010045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:17.013769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:17.150120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1248/2477] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:16 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:17.155604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:17.167862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32d2fb-709c-4433-a826-ef2214d18d9b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:17.256113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:17.256994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.257344 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.257617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.258284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:17.261118 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:17.372171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1230/2478] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:17 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:17.376887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:17.391319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1b26-d994-44d1-be19-826b79d3b638 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:17.406343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:17.407724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.408484 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.409159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.410569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:17.413112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:17.429934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1249/2479] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:17.431825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:17.448323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143b36a-5439-4fe9-b260-47d6679aa656 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:17.457239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b75215-ef81-4e72-a736-9a6826b145b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:17.458363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b75215-ef81-4e72-a736-9a6826b145b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.459746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b75215-ef81-4e72-a736-9a6826b145b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.460025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b75215-ef81-4e72-a736-9a6826b145b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.461328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b75215-ef81-4e72-a736-9a6826b145b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:17.464191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b75215-ef81-4e72-a736-9a6826b145b9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:17.483748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1231/2480] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:17.489218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:17.506483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18131a3e-71eb-4f46-a006-d8d9fad6517b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:17.513449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:17.515015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.515015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.515303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] PATH_INFO: `/v3/projects/b5751c51f943406e81a326907e7b9570/users/0420b49d082c42a5b7d4249cd4070c6e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.534415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5751c51f943406e81a326907e7b9570, user_id=0420b49d082c42a5b7d4249cd4070c6e, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:17.537785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:17.606589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:17.686999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1250/2481] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:17 2022] PUT /identity/v3/projects/b5751c51f943406e81a326907e7b9570/users/0420b49d082c42a5b7d4249cd4070c6e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:17.699394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.699676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.699944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.748023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] MFA Rules not processed for user `0420b49d082c42a5b7d4249cd4070c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:17.749731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:17.846536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1232/2482] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:17.851497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:17.960888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492249ae-7ae2-44d4-9f04-3a714fe6cdcf admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:17.972900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:17.973677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:17.973917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:17.974202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:17.974894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:17.976998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:18.023362 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:18.024996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:18.043545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50067b6-667b-481f-92a6-e34adb76d2d9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:18.051185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:18.051857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.052136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.052453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] PATH_INFO: `/v3/users/4cf154372a7b4d489e93908b681faf69` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.055352 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf154372a7b4d489e93908b681faf69)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:18.056600 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:18.221130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1251/2483] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:17 2022] POST /identity/v3/projects => generated 377 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:18.221130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:18.310237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9800b-1503-483d-bf45-6bb5e8cf4ce8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:18.326505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:18.328553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.329015 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.329450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.330270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:18.333180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:18.433056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf154372a7b4d489e93908b681faf69'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:18.486044 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1252/2484] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:18 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:18.491727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:18.515268 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82870e-6bd7-441d-bdbe-d48e0d3b03a2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:18.535984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:18.537735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.538026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.538302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.539043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:18.543855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:18.593197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1253/2485] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:52:18.593197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:18.593197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd0120-627c-427f-ad2d-94073f549083 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:18.593197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:18.593764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.593764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.594131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.594585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:18.596568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:18.616748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1254/2486] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:18.618969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:18.625346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf154372a7b4d489e93908b681faf69'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:18.626284 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf154372a7b4d489e93908b681faf69'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:18.636934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d57e00-4a09-413f-a786-2c18f9cdd621 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:18.640582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf154372a7b4d489e93908b681faf69'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:18.647729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae2696-8f79-48fb-8cdd-dfb8373d8b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:18.648253 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae2696-8f79-48fb-8cdd-dfb8373d8b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.648819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae2696-8f79-48fb-8cdd-dfb8373d8b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.649111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae2696-8f79-48fb-8cdd-dfb8373d8b90 admin admin] PATH_INFO: `/v3/projects/338bb484c2a64958bdf1918c4bd69eb0/users/8966cb98f57a41dc997d29fadcec61eb/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.672828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-433a1c9e-3db8-4e9f-b851-2ace16e536b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf154372a7b4d489e93908b681faf69'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:18.677171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae2696-8f79-48fb-8cdd-dfb8373d8b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338bb484c2a64958bdf1918c4bd69eb0, user_id=8966cb98f57a41dc997d29fadcec61eb, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:18.681580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae2696-8f79-48fb-8cdd-dfb8373d8b90 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:18.684612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1233/2487] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:18 2022] DELETE /identity/v3/users/4cf154372a7b4d489e93908b681faf69 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:18.690693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.690981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.691346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.737529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:18.738791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:18.831867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1234/2488] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:18.847840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:18.851236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:18.865103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1255/2489] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:18 2022] PUT /identity/v3/projects/338bb484c2a64958bdf1918c4bd69eb0/users/8966cb98f57a41dc997d29fadcec61eb/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:18.867398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:18.867786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:18.868158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:18.904771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] MFA Rules not processed for user `8966cb98f57a41dc997d29fadcec61eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:18.906241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:18.951855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222889b6-6988-45d4-af51-f1762fc576c5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:19.034114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1256/2490] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:19.038632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:19.040307 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:19.046784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:19.047615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:19.047773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:19.047932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:19.133869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac467019-ccbf-4de1-9d26-4b57c45de6ce None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:19.143115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:19.143716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:19.145058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:19.145342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:19.146089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:19.146593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:19.147524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:19.149893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:19.158825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1235/2491] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:19.293811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1257/2492] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:19 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:19.295109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:19.309039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b7017-39ff-43f5-9591-5e0a956f93bb service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:19.384099 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:19.384716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:19.384990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:19.385192 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:19.386477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:19.387879 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:19.575715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1236/2493] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:19 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:19.580134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:19.594405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e62c1-4de8-4e70-bc86-62da58e00111 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:19.600278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:19.601197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:19.601197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:19.601591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:19.602252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:19.603958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:19.620754 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1258/2494] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:19.623469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:19.650617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c7588-5c53-4fca-b475-80f236990048 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:19.656681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:19.657518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:19.657984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:19.658338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] PATH_INFO: `/v3/projects/aa2ffe57faf14903b4d63a0a42a2cde4/users/af3619612b784d45a1cb340320ff5e12/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:19.680995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2ffe57faf14903b4d63a0a42a2cde4, user_id=af3619612b784d45a1cb340320ff5e12, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:19.686052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:19.856974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1237/2495] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:19 2022] PUT /identity/v3/projects/aa2ffe57faf14903b4d63a0a42a2cde4/users/af3619612b784d45a1cb340320ff5e12/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:19.858238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:19.871935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397e77-77ab-436d-8ee4-d3631df8e948 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:19.982280 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5aa61-9490-4ad1-a488-660f20835464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:19.983155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5aa61-9490-4ad1-a488-660f20835464 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:19.983436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5aa61-9490-4ad1-a488-660f20835464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:19.984087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5aa61-9490-4ad1-a488-660f20835464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:19.985218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5aa61-9490-4ad1-a488-660f20835464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:19.995876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5aa61-9490-4ad1-a488-660f20835464 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:20.008923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1259/2496] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:52:20.013074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:20.026843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7f946-295e-4ead-bf8f-f57544a93990 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:20.034334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:20.035018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:20.035334 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:20.035836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] PATH_INFO: `/v3/projects/aa2ffe57faf14903b4d63a0a42a2cde4/users/af3619612b784d45a1cb340320ff5e12/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:20.038553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2ffe57faf14903b4d63a0a42a2cde4, user_id=af3619612b784d45a1cb340320ff5e12, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:20.041676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:20.054363 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1238/2497] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:20 2022] PUT /identity/v3/projects/aa2ffe57faf14903b4d63a0a42a2cde4/users/af3619612b784d45a1cb340320ff5e12/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:20.076820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:20.082130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:20.082130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:20.133599 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:20.135754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:20.172334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] MFA Rules not processed for user `af3619612b784d45a1cb340320ff5e12`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:20.175998 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:20.300282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7472-1b1b-45bd-bd0b-dffd07ce724f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:20.307141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:20.307797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:20.308076 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:20.308382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] PATH_INFO: `/v3/projects/f30a5fa4bd204a2ea790b9afa6f4e663` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:20.309665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30a5fa4bd204a2ea790b9afa6f4e663)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:20.311665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:20.341286 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1260/2498] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:20.356059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:20.358705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:20.390974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:20.442900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db224a1-30ef-4285-868a-857edaef4f72 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:20.453063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:20.453921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:20.454414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:20.454917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:20.457294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:20.514752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30a5fa4bd204a2ea790b9afa6f4e663', 'request_id': 'req-703fe522-f214-48c1-8e57-ddbe5fdbd925'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:20.567553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:20.569433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:20.583379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1261/2499] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:20.650779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30a5fa4bd204a2ea790b9afa6f4e663', 'request_id': 'req-703fe522-f214-48c1-8e57-ddbe5fdbd925'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:20.658209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30a5fa4bd204a2ea790b9afa6f4e663', 'request_id': 'req-703fe522-f214-48c1-8e57-ddbe5fdbd925'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:20.675787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] The token cache is being invalidate because project f30a5fa4bd204a2ea790b9afa6f4e663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:20.676449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:20.678663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:20.685868 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1239/2500] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:20 2022] DELETE /identity/v3/projects/f30a5fa4bd204a2ea790b9afa6f4e663 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:20.687138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:20.710858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1209f-775b-4d96-937a-904ff56081f7 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:20.800691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:20.801344 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:20.801629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:20.801970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] PATH_INFO: `/v3/users/288659d4cdef47948be0e4cf397d5cef` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:20.804162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] RBAC: Authorizing `identity:delete_user(user_id=288659d4cdef47948be0e4cf397d5cef)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:20.807151 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:21.379982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288659d4cdef47948be0e4cf397d5cef'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:21.380971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288659d4cdef47948be0e4cf397d5cef'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:21.389953 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288659d4cdef47948be0e4cf397d5cef'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:21.548590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288659d4cdef47948be0e4cf397d5cef'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:21.549407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fba96f88-f162-4662-a816-809a0c42505b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288659d4cdef47948be0e4cf397d5cef'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:21.577059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1262/2501] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:20 2022] DELETE /identity/v3/users/288659d4cdef47948be0e4cf397d5cef => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:21.811377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:22.399897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:22.399897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:22.421315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703fe522-f214-48c1-8e57-ddbe5fdbd925 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:22.509948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:22.510635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:22.510915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:22.511191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] PATH_INFO: `/v3/projects/22ef9ab19bca4838b582fe42df6af338` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:22.512682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ef9ab19bca4838b582fe42df6af338)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:22.516750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:22.782581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ef9ab19bca4838b582fe42df6af338', 'request_id': 'req-e0073494-4d85-48f7-b059-d6236e1ce7d7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:22.855077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:22.855887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:22.856404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:22.921582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] MFA Rules not processed for user `aaa4ddbee1034946a1f4dfe751a2b5cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:22.922914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:22.945836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ef9ab19bca4838b582fe42df6af338', 'request_id': 'req-e0073494-4d85-48f7-b059-d6236e1ce7d7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:22.960203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ef9ab19bca4838b582fe42df6af338', 'request_id': 'req-e0073494-4d85-48f7-b059-d6236e1ce7d7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:22.976136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] The token cache is being invalidate because project 22ef9ab19bca4838b582fe42df6af338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:22.976612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:22.977861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e0073494-4d85-48f7-b059-d6236e1ce7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:22.984927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1240/2502] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:22 2022] DELETE /identity/v3/projects/22ef9ab19bca4838b582fe42df6af338 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:22.987056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd6f51-55e7-466a-9491-9be0f922413d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:22.987567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd6f51-55e7-466a-9491-9be0f922413d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:22.988141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd6f51-55e7-466a-9491-9be0f922413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.018994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-44cd6f51-55e7-466a-9491-9be0f922413d None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:23.020703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44cd6f51-55e7-466a-9491-9be0f922413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:23.039248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1263/2503] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:23.047790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.048538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:23.147427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727e9b1-b284-4c04-bf4c-21f9cd349501 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.148654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1241/2504] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:23.158164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.158542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.158884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.159183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.159463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.159589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.160108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.160229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.163353 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.199761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:23.201198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:23.210029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1242/2505] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:23.215910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.227517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a900a-a097-4ed1-9270-b510544ebdfc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.234797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.235223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.235223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.235432 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.236115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:23.236449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.238080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.253024 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1243/2506] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:23.257983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.291814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fd610-6a72-4174-aee2-274c5badd4b3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.321707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1264/2507] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:23.322151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.322151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.322151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.322151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.322697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.325941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.326256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.449853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc246dd3-31f8-4abc-a9f1-3ef53d2e8857 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.458258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.459444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.459955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.460536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.461873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.463717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.617234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1244/2508] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:23.622738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.622738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:23.705791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5a6dc-0435-4bcd-9e6a-03e1b4245821 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.710901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.711662 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.712018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.712329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.714276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:23.730806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1265/2509] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:23 2022] POST /identity/v3/users => generated 295 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:23.731843 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.745522 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edc08c-0bc5-472a-bf1e-9a830861f96f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.808777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.810397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.822377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1245/2510] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:23.823374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.838183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.838963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.839318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.839705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.840701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.842987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.921884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e05d30-278a-4ad5-ab3a-541f9a4e4913 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.931928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.932685 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.933024 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.933354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.934170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:23.937067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:23.949884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1246/2511] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:23.965002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:23.980412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fde10-ccf3-4858-b42b-1dcbdff5190d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:23.984564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1266/2512] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:23 2022] POST /identity/v3/users => generated 289 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:23.986187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:23.986952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:23.987299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:23.987665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] PATH_INFO: `/v3/projects/8e0f1d8b8e0340f4b47820ef3de2ca1a/users/eeb945354f9d483784dedda9155cdbd4/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:23.990468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.011462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c2a59-b214-4b15-8529-b2144c45dcc0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.022952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.024543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.024917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.025256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.026029 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.026991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f1d8b8e0340f4b47820ef3de2ca1a, user_id=eeb945354f9d483784dedda9155cdbd4, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.028447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.031048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.042757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1267/2513] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:52:24.046826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.058664 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab0154-0b6e-43ed-9f07-5c7482a019ee admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.064676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.065283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.065562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.065834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.066572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.068203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.080095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1268/2514] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:24.087007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.099390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18893a-e28d-4638-86eb-bf33eaff3e16 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.105451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.106131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.107914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.108208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] PATH_INFO: `/v3/projects/7e486064976041c1bc944b795cc129b5/users/200fe0a2c1814150a1ced34e35aca65a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.126933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e486064976041c1bc944b795cc129b5, user_id=200fe0a2c1814150a1ced34e35aca65a, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.129818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.161312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1247/2515] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:23 2022] PUT /identity/v3/projects/8e0f1d8b8e0340f4b47820ef3de2ca1a/users/eeb945354f9d483784dedda9155cdbd4/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:24.167291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.276576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1269/2516] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:24 2022] PUT /identity/v3/projects/7e486064976041c1bc944b795cc129b5/users/200fe0a2c1814150a1ced34e35aca65a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:24.295785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.297728 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:24.299540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec9e98-890f-42e0-9cb5-7e2051a48147 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.404032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884d7e4-3af0-4a9a-91d9-8fe368342475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.405086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884d7e4-3af0-4a9a-91d9-8fe368342475 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.405636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884d7e4-3af0-4a9a-91d9-8fe368342475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.405872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884d7e4-3af0-4a9a-91d9-8fe368342475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.406857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884d7e4-3af0-4a9a-91d9-8fe368342475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.409624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884d7e4-3af0-4a9a-91d9-8fe368342475 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.436182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1248/2517] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:24.441326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.441326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.441326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.462666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394974e1-9ab8-4d2d-843f-75e984636c06 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.473075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.475588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.475588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.475588 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.478722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:24.483172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] MFA Rules not processed for user `200fe0a2c1814150a1ced34e35aca65a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:24.484882 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:24.496292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.499731 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.524020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1270/2518] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:24.526018 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.554281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe4d0e-29f8-4938-9aac-36b03acb87b9 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.583366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.591529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.591846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.592119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] PATH_INFO: `/v3/projects/8e0f1d8b8e0340f4b47820ef3de2ca1a/users/eeb945354f9d483784dedda9155cdbd4/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.598771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0f1d8b8e0340f4b47820ef3de2ca1a, user_id=eeb945354f9d483784dedda9155cdbd4, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.601761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.619387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1271/2519] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:24 2022] PUT /identity/v3/projects/8e0f1d8b8e0340f4b47820ef3de2ca1a/users/eeb945354f9d483784dedda9155cdbd4/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:24.628485 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1249/2520] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:24.633595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.726125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0f9a7-9bf9-471d-851e-313929575c20 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:24.732754 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:24.733386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:24.733665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:24.733940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:24.734575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:24.736554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:24.923753 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1272/2521] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:24 2022] POST /identity/v3/projects => generated 373 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:24.931911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:24.946729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc8eb-7a56-44e2-81c5-ca16d274dc5e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:25.035361 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:25.036034 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:25.036296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:25.036598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:25.037934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:25.040238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:25.314399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:25.326114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1250/2522] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:24 2022] POST /identity/v3/users => generated 293 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:25.326746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:25.357141 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167722a-166e-448e-bc9a-1eab0a06ca81 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:25.373024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:25.373922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:25.374432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:25.375082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:25.375980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:25.377923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:25.393665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1273/2523] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:25.412393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:25.430001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051d1ff-79bd-40af-9030-244921789fd7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:25.436698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:25.443442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:25.443442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:25.443442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] PATH_INFO: `/v3/projects/faa3894a79a0459ab90142f4facf4e3f/users/7816b2da1a3b4bff92299a85fb6df2f8/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:25.470190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa3894a79a0459ab90142f4facf4e3f, user_id=7816b2da1a3b4bff92299a85fb6df2f8, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:25.473318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:25.614697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1251/2524] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:25 2022] PUT /identity/v3/projects/faa3894a79a0459ab90142f4facf4e3f/users/7816b2da1a3b4bff92299a85fb6df2f8/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:25.620937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:25.642100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8c9b7-b9bf-473c-b4d2-c5618ca0407e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:25.790740 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dd781-df5f-4850-bb22-94ad08c2a9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:25.799537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dd781-df5f-4850-bb22-94ad08c2a9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:25.799537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dd781-df5f-4850-bb22-94ad08c2a9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:25.799537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dd781-df5f-4850-bb22-94ad08c2a9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:25.799537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dd781-df5f-4850-bb22-94ad08c2a9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:25.801201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dd781-df5f-4850-bb22-94ad08c2a9f4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:25.818037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1274/2525] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:25.823158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:25.836892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe763c4-b2d6-435b-9e7c-4405f583c121 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:25.843484 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:25.844169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:25.844571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:25.844790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] PATH_INFO: `/v3/projects/faa3894a79a0459ab90142f4facf4e3f/users/7816b2da1a3b4bff92299a85fb6df2f8/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:25.846744 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa3894a79a0459ab90142f4facf4e3f, user_id=7816b2da1a3b4bff92299a85fb6df2f8, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:25.851123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:25.865972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1252/2526] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:25 2022] PUT /identity/v3/projects/faa3894a79a0459ab90142f4facf4e3f/users/7816b2da1a3b4bff92299a85fb6df2f8/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:25.879928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:25.880424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:25.880889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:25.947372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] MFA Rules not processed for user `7816b2da1a3b4bff92299a85fb6df2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:25.949933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:26.056537 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1275/2527] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:26.069820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:26.072392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:26.149144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3126b4b-8f64-4d03-954e-7ffe147be1e6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:26.159364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:26.160425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:26.160867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:26.161307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:26.235421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:26.236957 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:26.248666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1253/2528] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:26.461913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:26.481967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fa35f-a5a7-4213-9033-9858417bd6cb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:26.487485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:26.488142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:26.488478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:26.488742 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:26.491439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:26.585331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:26.587498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7012a68-8200-47ee-bb7f-024d9d561cbc service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:26.599579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1276/2529] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:26.646730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:28.772113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:29.104584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:29.107748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:29.229840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf7caa-5226-4fc3-aba5-461e5ba67869 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:29.237080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:29.238201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:29.238670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:29.239897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] PATH_INFO: `/v3/users/f2bcf5e4247a4e45a7fb2c0ba3b57d65` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:29.242291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2bcf5e4247a4e45a7fb2c0ba3b57d65)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:29.248818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:29.858343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2bcf5e4247a4e45a7fb2c0ba3b57d65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:30.018658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2bcf5e4247a4e45a7fb2c0ba3b57d65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:30.020151 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2bcf5e4247a4e45a7fb2c0ba3b57d65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:30.031457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2bcf5e4247a4e45a7fb2c0ba3b57d65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:30.046150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2bcf5e4247a4e45a7fb2c0ba3b57d65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:30.054928 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1254/2530] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:29 2022] DELETE /identity/v3/users/f2bcf5e4247a4e45a7fb2c0ba3b57d65 => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:30.063088 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:30.063739 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:30.064206 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:30.097274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:30.100759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:30.216063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1277/2531] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:30.230945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:30.232518 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:30.322436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df49b2-5ffb-4422-af72-6a97ca85e505 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:30.328166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717064db-edfe-4817-b1bd-ecb680ab5b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:30.329410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717064db-edfe-4817-b1bd-ecb680ab5b8e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:30.329828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717064db-edfe-4817-b1bd-ecb680ab5b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:30.330262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717064db-edfe-4817-b1bd-ecb680ab5b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:30.343208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717064db-edfe-4817-b1bd-ecb680ab5b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:30.345604 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717064db-edfe-4817-b1bd-ecb680ab5b8e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:30.362023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1255/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:31.311785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:31.313411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:31.331386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015de87-7b52-4f6e-8c9a-b8eaf8de10e6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:31.341626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:31.342680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:31.343112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:31.344000 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] PATH_INFO: `/v3/projects/4ed4a384fa1f42d1b644ea9cfb291b60` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:31.345641 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed4a384fa1f42d1b644ea9cfb291b60)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:31.350391 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:31.549233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:31.637386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed4a384fa1f42d1b644ea9cfb291b60', 'request_id': 'req-bf48a2a2-d518-4d58-b0e4-848cb28e2401'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:31.645291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed4a384fa1f42d1b644ea9cfb291b60', 'request_id': 'req-bf48a2a2-d518-4d58-b0e4-848cb28e2401'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:31.835184 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed4a384fa1f42d1b644ea9cfb291b60', 'request_id': 'req-bf48a2a2-d518-4d58-b0e4-848cb28e2401'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:31.854379 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] The token cache is being invalidate because project 4ed4a384fa1f42d1b644ea9cfb291b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:31.854952 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:31.856575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-bf48a2a2-d518-4d58-b0e4-848cb28e2401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:31.859996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1278/2533] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:31 2022] DELETE /identity/v3/projects/4ed4a384fa1f42d1b644ea9cfb291b60 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:31.868145 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:31.868466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:31.868974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:31.906665 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:31.909201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:31.999712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1256/2534] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:32.009735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.010213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.010673 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.033614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:32.034968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:32.046400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1279/2535] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:32.051318 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:32.066587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638bc6f-e8d9-4e65-ba8c-7594b7be449e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:32.072909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:32.073535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.077431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.077431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.077431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:32.077431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:32.077431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:32.091130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1257/2536] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:52:32.096323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:32.107134 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488daec5-766f-49c8-8d28-e00433ce9b11 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:32.112575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:32.113199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.113478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.113748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.114397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:32.116473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:32.389559 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1280/2537] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:32 2022] POST /identity/v3/projects => generated 375 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:32.394870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:32.429155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa98737-b6e2-4649-8755-ef51d74963bd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:32.533356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:32.556514 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:32.557542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.557907 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.558248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.559004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:32.561441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:32.715497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1258/2538] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:32 2022] POST /identity/v3/users => generated 288 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:32.716527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:32.737775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9bd41-01c6-4327-b31b-f23d34fa0f54 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:32.746345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:32.746970 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.747247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.747520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.748177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:32.749936 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:32.763851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1281/2539] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:32.770066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:32.787231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297adaa6-70b3-42d8-8450-4a4452318f51 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:32.795746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aab184-4fc5-4d51-b7b7-c80843805be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:32.796595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aab184-4fc5-4d51-b7b7-c80843805be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.796988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aab184-4fc5-4d51-b7b7-c80843805be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.797328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aab184-4fc5-4d51-b7b7-c80843805be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.798098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aab184-4fc5-4d51-b7b7-c80843805be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:32.801205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aab184-4fc5-4d51-b7b7-c80843805be3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:32.824759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1259/2540] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:32.829536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:32.846922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef8aca-540c-4737-b176-d09cb9987d03 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:32.861841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:32.862665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:32.862971 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:32.863480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] PATH_INFO: `/v3/projects/f1843a2e87ac455aa980926e2a0cecad/users/5d92b293c374426885ef3f1157e7f6a3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:32.905413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1843a2e87ac455aa980926e2a0cecad, user_id=5d92b293c374426885ef3f1157e7f6a3, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:32.910979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:33.080158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1282/2541] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:32 2022] PUT /identity/v3/projects/f1843a2e87ac455aa980926e2a0cecad/users/5d92b293c374426885ef3f1157e7f6a3/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:33.092782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:33.093065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:33.093557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:33.162717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] MFA Rules not processed for user `5d92b293c374426885ef3f1157e7f6a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:33.164259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:33.274437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1260/2542] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:33.280974 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:33.417284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d85d8c-c995-4b1b-90b4-4a4442a25e90 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:33.425525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:33.426349 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:33.426926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:33.427200 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:33.428848 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:33.431061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:33.652401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1283/2543] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:33 2022] POST /identity/v3/projects => generated 375 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:33.654913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:33.695179 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136ee84-a8de-4885-a651-be4673835b20 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:33.766536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:33.769677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:33.873012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:33.874116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:33.874680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:33.875057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:33.876139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:33.886945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:33.938634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0762ad-7a52-449a-8301-03d376447201 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:33.948027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:33.949356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:33.949841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:33.950370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:33.958485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:34.091637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1261/2544] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:33 2022] POST /identity/v3/users => generated 294 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:34.092786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:34.094801 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:34.113392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:34.116784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2258b3-cbcf-48ea-9df0-7e9b1b3cf1a5 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:34.122090 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cde2f7-a3af-44ec-beb9-ff9c2c4c7dca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:34.130281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:34.131926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1284/2545] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:34.133327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:34.133680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:34.133866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:34.134064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:34.134283 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:34.134859 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] PATH_INFO: `/v3/users/0420b49d082c42a5b7d4249cd4070c6e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:34.137530 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0420b49d082c42a5b7d4249cd4070c6e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:34.144303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:34.194264 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] MFA Rules not processed for user `eeb945354f9d483784dedda9155cdbd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:34.196466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:34.246354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:34.335069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1285/2546] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:34.336025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:34.363192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70df7-40f7-4136-a56b-876883bb007d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:34.375706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:34.377164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:34.377717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:34.378166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:34.379303 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:34.383171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:34.415709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1286/2547] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:34.415709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:34.415709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:34.536616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0420b49d082c42a5b7d4249cd4070c6e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:34.582153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08124efb-9421-4b60-80f2-48f7cdc7ddf3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:34.592852 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:34.593539 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:34.593831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:34.594069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:34.596148 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:34.680723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0420b49d082c42a5b7d4249cd4070c6e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:34.681764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0420b49d082c42a5b7d4249cd4070c6e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:34.700774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:34.706629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:34.712164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0420b49d082c42a5b7d4249cd4070c6e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:34.722234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1287/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:34.723589 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:34.728188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30991e13-207f-449f-84ac-66cbfe14fc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0420b49d082c42a5b7d4249cd4070c6e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:34.750569 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c00902-f59e-4d5e-9aaf-c194628f999b service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:34.760383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1262/2549] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:34 2022] DELETE /identity/v3/users/0420b49d082c42a5b7d4249cd4070c6e => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:34.760383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:34.760383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:34.760383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:34.843473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:34.847458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:34.923964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:34.925764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:34.926762 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:34.927582 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] PATH_INFO: `/v3/projects/5ec79e40641c4f8aace4f984ca5b4650/users/4a67425c07ca4ccd8b9bf57c6b9dbfca/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:34.961073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec79e40641c4f8aace4f984ca5b4650, user_id=4a67425c07ca4ccd8b9bf57c6b9dbfca, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:34.965575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:35.040055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1263/2550] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:35.048276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:35.058065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:35.058569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:35.159906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1288/2551] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:34 2022] PUT /identity/v3/projects/5ec79e40641c4f8aace4f984ca5b4650/users/4a67425c07ca4ccd8b9bf57c6b9dbfca/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:35.167647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:35.170528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:35.202343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f48eaa-e02e-478d-80f6-0d64bac5189c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:35.284702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:35.288442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:35.288930 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:35.289359 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:35.306571 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57054f56-bd44-40dc-ad8a-02dc24a9531a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:35.308935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:35.311884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:35.318197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c5dfa-968a-459b-985d-dec722d38e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:35.319906 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c5dfa-968a-459b-985d-dec722d38e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:35.320578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c5dfa-968a-459b-985d-dec722d38e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:35.320923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c5dfa-968a-459b-985d-dec722d38e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:35.321657 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c5dfa-968a-459b-985d-dec722d38e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:35.325063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c5dfa-968a-459b-985d-dec722d38e73 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:35.332043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1264/2552] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:35.353270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1289/2553] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:52:35.354519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:35.375701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f0238-11b2-49fe-81bb-c739d299017d service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:35.390878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:35.391357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:35.391648 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:35.392098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] PATH_INFO: `/v3/projects/5ec79e40641c4f8aace4f984ca5b4650/users/4a67425c07ca4ccd8b9bf57c6b9dbfca/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:35.396223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec79e40641c4f8aace4f984ca5b4650, user_id=4a67425c07ca4ccd8b9bf57c6b9dbfca, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:35.403086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:35.426965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1265/2554] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:35 2022] PUT /identity/v3/projects/5ec79e40641c4f8aace4f984ca5b4650/users/4a67425c07ca4ccd8b9bf57c6b9dbfca/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:35.446136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:35.446629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:35.447070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:35.511284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] MFA Rules not processed for user `4a67425c07ca4ccd8b9bf57c6b9dbfca`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:35.513890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:35.677567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1290/2555] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:35.697978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:35.700022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:35.799758 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c0e91-1dd6-4402-b138-fdfad5242aa7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:35.808315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:35.809338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:35.809782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:35.810219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:35.813141 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:35.941491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:35.944436 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:35.956950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1266/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:36.214655 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:36.217905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:36.338837 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91aa5d5-3916-415e-bc9d-38956f732d00 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:36.345457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:36.346130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:36.346416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:36.346693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] PATH_INFO: `/v3/projects/b5751c51f943406e81a326907e7b9570` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:36.348039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5751c51f943406e81a326907e7b9570)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:36.351004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:36.676132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5751c51f943406e81a326907e7b9570', 'request_id': 'req-f3441eda-99bf-4d21-878a-b5e9d9de5a75'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:36.685150 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5751c51f943406e81a326907e7b9570', 'request_id': 'req-f3441eda-99bf-4d21-878a-b5e9d9de5a75'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:36.812719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5751c51f943406e81a326907e7b9570', 'request_id': 'req-f3441eda-99bf-4d21-878a-b5e9d9de5a75'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:36.838039 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] The token cache is being invalidate because project b5751c51f943406e81a326907e7b9570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:36.838879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:36.840381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:36.843810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1291/2557] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:36 2022] DELETE /identity/v3/projects/b5751c51f943406e81a326907e7b9570 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:36.850923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:36.871421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045a77-ea83-465c-a512-d55081f1c3d5 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:36.969107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:36.969789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:36.970063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:36.970332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] PATH_INFO: `/v3/users/8966cb98f57a41dc997d29fadcec61eb` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:36.971561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8966cb98f57a41dc997d29fadcec61eb)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:36.973849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:37.498002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8966cb98f57a41dc997d29fadcec61eb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:37.714419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8966cb98f57a41dc997d29fadcec61eb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:37.715244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8966cb98f57a41dc997d29fadcec61eb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:37.733422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8966cb98f57a41dc997d29fadcec61eb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:37.743086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8966cb98f57a41dc997d29fadcec61eb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:37.760614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1267/2558] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:36 2022] DELETE /identity/v3/users/8966cb98f57a41dc997d29fadcec61eb => generated 0 bytes in 910 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:38.733262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:38.735846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:38.751652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3441eda-99bf-4d21-878a-b5e9d9de5a75 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:38.891066 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:38.893310 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:38.894785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:38.895423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] PATH_INFO: `/v3/projects/338bb484c2a64958bdf1918c4bd69eb0` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:38.905146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=338bb484c2a64958bdf1918c4bd69eb0)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:38.908137 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:39.187633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338bb484c2a64958bdf1918c4bd69eb0', 'request_id': 'req-aafb467c-5c32-46eb-917d-a96cc7829057'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:39.207729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338bb484c2a64958bdf1918c4bd69eb0', 'request_id': 'req-aafb467c-5c32-46eb-917d-a96cc7829057'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:39.485968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338bb484c2a64958bdf1918c4bd69eb0', 'request_id': 'req-aafb467c-5c32-46eb-917d-a96cc7829057'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:39.500009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] The token cache is being invalidate because project 338bb484c2a64958bdf1918c4bd69eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:39.500333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:39.501174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:39.502968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1292/2559] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:38 2022] DELETE /identity/v3/projects/338bb484c2a64958bdf1918c4bd69eb0 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:39.508673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:39.517372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:39.524319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea90b3f-90a8-41d7-b57f-f5e01fbc68ae admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:39.592848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:39.593499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:39.593779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:39.594077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] PATH_INFO: `/v3/users/af3619612b784d45a1cb340320ff5e12` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:39.595253 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3619612b784d45a1cb340320ff5e12)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:39.597291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:40.080324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3619612b784d45a1cb340320ff5e12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:40.269417 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3619612b784d45a1cb340320ff5e12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:40.270224 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3619612b784d45a1cb340320ff5e12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:40.297115 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3619612b784d45a1cb340320ff5e12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:40.326048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3619612b784d45a1cb340320ff5e12'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:40.335951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1268/2560] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:39 2022] DELETE /identity/v3/users/af3619612b784d45a1cb340320ff5e12 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:41.138077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:41.221101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:41.222497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:41.247800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb467c-5c32-46eb-917d-a96cc7829057 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:41.338149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:41.338781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:41.339059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:41.339332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] PATH_INFO: `/v3/projects/aa2ffe57faf14903b4d63a0a42a2cde4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:41.340515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2ffe57faf14903b4d63a0a42a2cde4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:41.342420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:41.508449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:41.511753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:41.528892 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195c63b-14bd-4fe3-bbd2-7d4ce98c8a2b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:41.539666 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:41.540488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:41.540888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:41.541385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] PATH_INFO: `/v3/users/5d92b293c374426885ef3f1157e7f6a3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:41.543086 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d92b293c374426885ef3f1157e7f6a3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:41.545162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:41.652692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2ffe57faf14903b4d63a0a42a2cde4', 'request_id': 'req-56898db5-fc0a-460a-90d9-5876fe8effdb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:41.664222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2ffe57faf14903b4d63a0a42a2cde4', 'request_id': 'req-56898db5-fc0a-460a-90d9-5876fe8effdb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:41.925646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2ffe57faf14903b4d63a0a42a2cde4', 'request_id': 'req-56898db5-fc0a-460a-90d9-5876fe8effdb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:41.942932 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] The token cache is being invalidate because project aa2ffe57faf14903b4d63a0a42a2cde4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:41.947458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:41.947458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-56898db5-fc0a-460a-90d9-5876fe8effdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:41.947458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1293/2561] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:41 2022] DELETE /identity/v3/projects/aa2ffe57faf14903b4d63a0a42a2cde4 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:42.081990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d92b293c374426885ef3f1157e7f6a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:42.249165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d92b293c374426885ef3f1157e7f6a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:42.249789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d92b293c374426885ef3f1157e7f6a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:42.259028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d92b293c374426885ef3f1157e7f6a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:42.300372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d92b293c374426885ef3f1157e7f6a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:42.300372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1269/2562] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:41 2022] DELETE /identity/v3/users/5d92b293c374426885ef3f1157e7f6a3 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:42.302254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:42.302254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:42.302254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:42.312908 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:42.314201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:42.428492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1294/2563] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:42.435213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:42.437701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:42.528613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ec7e9-3201-443b-a9ae-4e3bbc4ff49e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:42.532821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:42.533443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:42.533717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:42.533984 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:42.548458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:42.549960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:42.562325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1270/2564] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:43.443877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:43.453628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:43.455726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:43.475171 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ee6f0-6d80-4417-95e1-6c65b47fd69c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:43.481829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:43.483156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:43.483436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:43.483707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] PATH_INFO: `/v3/projects/f1843a2e87ac455aa980926e2a0cecad` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:43.485464 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1843a2e87ac455aa980926e2a0cecad)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:43.487476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:43.739834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1843a2e87ac455aa980926e2a0cecad', 'request_id': 'req-635b3dd3-7009-4025-a5bd-c4a5030141c3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:43.752316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1843a2e87ac455aa980926e2a0cecad', 'request_id': 'req-635b3dd3-7009-4025-a5bd-c4a5030141c3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:43.962794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1843a2e87ac455aa980926e2a0cecad', 'request_id': 'req-635b3dd3-7009-4025-a5bd-c4a5030141c3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:43.986239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] The token cache is being invalidate because project f1843a2e87ac455aa980926e2a0cecad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:43.986643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:43.987451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:43.991215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1295/2565] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:43 2022] DELETE /identity/v3/projects/f1843a2e87ac455aa980926e2a0cecad => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:43.994955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:44.016764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25011a6-b8fc-4332-ab12-ddad1ff55f1c service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:44.079904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:44.080605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:44.080886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:44.081161 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] PATH_INFO: `/v3/users/4a67425c07ca4ccd8b9bf57c6b9dbfca` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:44.082446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a67425c07ca4ccd8b9bf57c6b9dbfca)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:44.085584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:44.588072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a67425c07ca4ccd8b9bf57c6b9dbfca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:44.602840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:44.797315 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a67425c07ca4ccd8b9bf57c6b9dbfca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:44.798264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a67425c07ca4ccd8b9bf57c6b9dbfca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:44.807266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a67425c07ca4ccd8b9bf57c6b9dbfca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:44.821002 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-96674fa7-6e53-4140-8277-4893e7fec72e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a67425c07ca4ccd8b9bf57c6b9dbfca'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:44.827198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1271/2566] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:52:43 2022] DELETE /identity/v3/users/4a67425c07ca4ccd8b9bf57c6b9dbfca => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:52:45.338581 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:45.340841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:45.356595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b3dd3-7009-4025-a5bd-c4a5030141c3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:45.426161 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:45.426797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:45.427072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:45.430345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] PATH_INFO: `/v3/projects/5ec79e40641c4f8aace4f984ca5b4650` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:45.430345 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec79e40641c4f8aace4f984ca5b4650)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:45.431190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:45.623212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec79e40641c4f8aace4f984ca5b4650', 'request_id': 'req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:45.630986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec79e40641c4f8aace4f984ca5b4650', 'request_id': 'req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:45.759884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec79e40641c4f8aace4f984ca5b4650', 'request_id': 'req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:45.773916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] The token cache is being invalidate because project 5ec79e40641c4f8aace4f984ca5b4650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:52:45.776515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:45.776515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9df8a84-d8fe-4137-a94c-cae5c2f5cc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:52:45.776984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1296/2567] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:52:45 2022] DELETE /identity/v3/projects/5ec79e40641c4f8aace4f984ca5b4650 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:45.781883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:45.782418 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:45.782542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:45.807456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:45.810496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:45.884831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1272/2568] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:45.893990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:45.894482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:45.894977 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:45.925871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:45.938393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:45.938393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1297/2569] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:52:45.939431 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:45.949467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca5e55-0d9f-47fd-a19e-0883ffd4a3c0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:45.954238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:45.954847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:45.955123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:45.955684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:45.956078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:52:45.956511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:45.958037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:45.971811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1273/2570] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:52:45.973102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:45.976775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:45.986487 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2d143-445c-47e3-ad18-880c600aa233 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:45.991534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:45.992136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:45.992438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:45.992711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:45.993370 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:45.995330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:46.189633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1298/2571] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:52:45 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:52:46.197911 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:46.210529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433cd9b-a07e-4059-a775-99be61999785 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:46.275054 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:46.275678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:46.275956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:46.276229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:46.276922 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:46.278941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:46.490787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1274/2572] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:52:46 2022] POST /identity/v3/users => generated 286 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:52:46.498561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:46.513688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946da22d-ff1e-404b-9fbb-e7c4f8f85790 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:46.520529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:46.521319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:46.523143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:46.524090 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:46.524855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:46.526635 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:46.539136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1299/2573] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:46.543545 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:46.554873 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290639-501a-47b6-8f53-d3606a081d3a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:46.559833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871405a6-a1c2-44c8-9b5c-d3d595fa6bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:46.560476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871405a6-a1c2-44c8-9b5c-d3d595fa6bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:46.560752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871405a6-a1c2-44c8-9b5c-d3d595fa6bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:46.561028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871405a6-a1c2-44c8-9b5c-d3d595fa6bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:46.561943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871405a6-a1c2-44c8-9b5c-d3d595fa6bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:46.563997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871405a6-a1c2-44c8-9b5c-d3d595fa6bfd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:46.577782 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1275/2574] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:52:46.582330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:46.595788 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a6542-4262-44d5-8883-e5af87ba579f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:46.600702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:46.601397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:46.601674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:46.601950 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] PATH_INFO: `/v3/projects/19ee69068eb94cf1b4a0d78f7d5e4a8b/users/651140ed5a704cde978be56393e0537e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:46.620374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ee69068eb94cf1b4a0d78f7d5e4a8b, user_id=651140ed5a704cde978be56393e0537e, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:46.623996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:46.797867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1300/2575] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:52:46 2022] PUT /identity/v3/projects/19ee69068eb94cf1b4a0d78f7d5e4a8b/users/651140ed5a704cde978be56393e0537e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:52:46.804814 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb197b-51dc-4ad2-8c90-412874d52554 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:46.805606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb197b-51dc-4ad2-8c90-412874d52554 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:46.805606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb197b-51dc-4ad2-8c90-412874d52554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:46.836952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-21eb197b-51dc-4ad2-8c90-412874d52554 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:46.838787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21eb197b-51dc-4ad2-8c90-412874d52554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:46.922477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1276/2576] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:46.934323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:46.936106 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:46.973364 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:47.019438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0cb38-2ef1-451a-a3a6-f903b08ffe26 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:47.025287 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:47.025958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:47.026427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:47.026613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:47.037395 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:47.040783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:47.053142 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1301/2577] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:52:49.333894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4ea3-8eeb-410d-b859-f4e5ca2cf959 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:49.334642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4ea3-8eeb-410d-b859-f4e5ca2cf959 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:49.334836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4ea3-8eeb-410d-b859-f4e5ca2cf959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:49.385560 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1d1b4ea3-8eeb-410d-b859-f4e5ca2cf959 None None] MFA Rules not processed for user `651140ed5a704cde978be56393e0537e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:49.387164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1d1b4ea3-8eeb-410d-b859-f4e5ca2cf959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:49.491481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1277/2578] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:49.514965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:49.516501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:49.601968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3cfe19-a2a5-43b8-8412-95ac99830bb2 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:49.606046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:49.606645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:49.606921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:49.607192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:49.616820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:49.618663 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:49.628451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1302/2579] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:50.160591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:50.894905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:51.247568 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:51.717497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:53.040126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:53.269917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:54.000091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:54.637285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:54.821273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:55.169948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:55.476638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:56.017437 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:56.029910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:56.029910 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:56.042774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] MFA Rules not processed for user `651140ed5a704cde978be56393e0537e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:52:56.044729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:56.052853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1278/2580] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:52:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:52:56.422069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:56.552834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:57.977224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:52:57.979210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:58.002555 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889db258-584d-471e-9416-0dd09522b0cc service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:52:58.010126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:52:58.012617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:52:58.012617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:52:58.012617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:52:58.017256 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:52:58.036700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:52:58.039530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:52:58.051830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1303/2581] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:52:58.113358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:58.227836 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:59.055008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:59.266028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:59.394735 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:59.880627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:52:59.919964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:00.347451 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:00.387831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:00.999383 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:03.091563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:03.106343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:03.106343 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:03.123133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0888008-68c6-4d34-bee9-774d23e66cd8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:03.128355 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:03.129201 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:03.129543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:03.129821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] PATH_INFO: `/v3/users/fc27435df46542a3ac37e743b6399387` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:03.132830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc27435df46542a3ac37e743b6399387)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:03.135981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:03.415433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc27435df46542a3ac37e743b6399387'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:03.612997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc27435df46542a3ac37e743b6399387'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:03.614456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc27435df46542a3ac37e743b6399387'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:03.627275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc27435df46542a3ac37e743b6399387'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:03.648064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35a6209f-3e78-4b76-9424-b5310ff6b05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc27435df46542a3ac37e743b6399387'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:03.661321 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1279/2582] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:03 2022] DELETE /identity/v3/users/fc27435df46542a3ac37e743b6399387 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:04.076712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:04.089476 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e4aec-7548-4213-bcf4-5e114be63f56 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:04.188912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:04.190300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:04.190300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:04.190300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] PATH_INFO: `/v3/projects/52f30f320fa04cdfa0e1d4ea9050119c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:04.191416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f30f320fa04cdfa0e1d4ea9050119c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:04.193424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:04.401143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:04.408040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f30f320fa04cdfa0e1d4ea9050119c', 'request_id': 'req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:04.418743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f30f320fa04cdfa0e1d4ea9050119c', 'request_id': 'req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:04.533014 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f30f320fa04cdfa0e1d4ea9050119c', 'request_id': 'req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:04.555681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] The token cache is being invalidate because project 52f30f320fa04cdfa0e1d4ea9050119c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:04.556453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:04.557248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2217d28b-8ad7-4cd9-9bb4-7b6fb8884d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:04.559496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1304/2583] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:04 2022] DELETE /identity/v3/projects/52f30f320fa04cdfa0e1d4ea9050119c => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:04.565330 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:04.565833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:04.566477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:04.591144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:04.592424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:04.680439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1280/2584] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:04.689399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:04.689945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:04.690244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:04.716584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:04.718293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:04.734419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1305/2585] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:04.739664 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:04.762944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310598b5-acd1-4329-ba95-21b1413ec1f0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:04.773755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:04.774390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:04.774663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:04.774931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:04.775596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:04.775894 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:04.777591 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:04.798857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1281/2586] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:53:04.809108 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:04.815566 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332db39d-139e-45fe-be15-76478e4a11f6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:04.822501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:04.823442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:04.823780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:04.824117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:04.825065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:04.827025 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:04.934688 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1306/2587] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:04 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:04.939727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:04.954195 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cbe8-c570-4323-b12a-68407922250e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:05.063466 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:05.063715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.064064 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.064538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.065556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:05.070429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:05.196661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1282/2588] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:04 2022] POST /identity/v3/users => generated 302 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:05.201704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:05.212774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:05.213884 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22549173-2790-411b-b5b4-1c84b8f4db2d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:05.219172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:05.220197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.220784 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.221666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.222259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:05.224301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:05.238627 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1307/2589] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:05.243377 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:05.262681 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a3ddd-11c6-49e8-9791-32bb8381c84b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:05.271704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd1979-83e7-42ed-b421-5c8ba1b88086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:05.273336 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd1979-83e7-42ed-b421-5c8ba1b88086 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.274093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd1979-83e7-42ed-b421-5c8ba1b88086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.274571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd1979-83e7-42ed-b421-5c8ba1b88086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.275738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd1979-83e7-42ed-b421-5c8ba1b88086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:05.278716 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd1979-83e7-42ed-b421-5c8ba1b88086 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:05.300452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1283/2590] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:05.302823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:05.314876 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90849e-d38d-4cb0-b101-8304b8b60da4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:05.320607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:05.321734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.322011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.322284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] PATH_INFO: `/v3/projects/6fb14b430d8d4d91804d791970096eec/users/9ec85d19f5024763829a22b26a07a2df/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.342293 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb14b430d8d4d91804d791970096eec, user_id=9ec85d19f5024763829a22b26a07a2df, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:05.345009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:05.481246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1308/2591] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:05 2022] PUT /identity/v3/projects/6fb14b430d8d4d91804d791970096eec/users/9ec85d19f5024763829a22b26a07a2df/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:05.494311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.494800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.495172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.535400 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] MFA Rules not processed for user `9ec85d19f5024763829a22b26a07a2df`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:05.536701 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:05.625420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1284/2592] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:05.631313 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:05.739042 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4406a-87ab-4212-9372-6c17c8669126 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:05.744915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:05.745785 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.746245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.746565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.747413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:05.749839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:05.871841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1309/2593] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:05 2022] POST /identity/v3/projects => generated 405 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:05.879238 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:05.898188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e81ead-9b2c-4412-8ff8-7b0478938874 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:05.982319 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:05.983205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:05.983846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:05.984223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:05.985067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:05.987182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:06.154245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1285/2594] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:05 2022] POST /identity/v3/users => generated 309 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:06.156120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:06.172628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca12c15-dcf2-4ad7-b5d3-d4f17b62aba0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:06.177831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:06.179080 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.179269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.179535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.180187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:06.182205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:06.195318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1310/2595] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:06.200824 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:06.214022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19e3e0-dd74-4282-a9c9-a3f10aa9436e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:06.219602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:06.220586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.220844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.221313 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] PATH_INFO: `/v3/projects/a5592966e3cc40c9b778c1e5a6194d30/users/bc901045d54e463a9eeffc80190c15e5/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.240190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5592966e3cc40c9b778c1e5a6194d30, user_id=bc901045d54e463a9eeffc80190c15e5, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:06.245306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:06.383268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1286/2596] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:06 2022] PUT /identity/v3/projects/a5592966e3cc40c9b778c1e5a6194d30/users/bc901045d54e463a9eeffc80190c15e5/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:06.388213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:06.408228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03cc8c-fccb-4395-bd91-00e4f2929866 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:06.465781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:06.481624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:06.508519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fe359-e5fe-4aaf-b7d2-1cf97a2e8098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:06.508680 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe359-e5fe-4aaf-b7d2-1cf97a2e8098 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.508964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe359-e5fe-4aaf-b7d2-1cf97a2e8098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.509246 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe359-e5fe-4aaf-b7d2-1cf97a2e8098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.510038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fe359-e5fe-4aaf-b7d2-1cf97a2e8098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:06.512082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fe359-e5fe-4aaf-b7d2-1cf97a2e8098 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:06.535204 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1311/2597] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:06.557944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:06.576721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace498de-d33f-4dc9-8e58-a1823ae48368 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:06.581903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03c71f-94f3-4d58-a252-afb65dcaaeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:06.582888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03c71f-94f3-4d58-a252-afb65dcaaeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.583166 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03c71f-94f3-4d58-a252-afb65dcaaeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.583433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03c71f-94f3-4d58-a252-afb65dcaaeb6 admin admin] PATH_INFO: `/v3/projects/a5592966e3cc40c9b778c1e5a6194d30/users/bc901045d54e463a9eeffc80190c15e5/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.585866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03c71f-94f3-4d58-a252-afb65dcaaeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5592966e3cc40c9b778c1e5a6194d30, user_id=bc901045d54e463a9eeffc80190c15e5, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:06.588937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03c71f-94f3-4d58-a252-afb65dcaaeb6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:06.604311 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1287/2598] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:06 2022] PUT /identity/v3/projects/a5592966e3cc40c9b778c1e5a6194d30/users/bc901045d54e463a9eeffc80190c15e5/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:06.616535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.617107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.617388 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.669675 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] MFA Rules not processed for user `bc901045d54e463a9eeffc80190c15e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:06.671113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:06.748972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.749632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.749632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.772126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] MFA Rules not processed for user `aaa4ddbee1034946a1f4dfe751a2b5cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:06.773619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:06.785686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1312/2599] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:06.811826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:06.813258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:06.863627 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1288/2600] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:06.882729 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9e583-a62d-4209-a1e0-ef1fb8e503ad None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:06.888989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:06.890012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:06.890889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:06.891226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:06.906643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:06.908126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:06.920071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1313/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:06 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:07.102258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:07.206529 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55356ab3-8ae0-446a-a699-1420e2796e9f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:07.213570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:07.214440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:07.214721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:07.214991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] PATH_INFO: `/v3/users/9ec85d19f5024763829a22b26a07a2df` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:07.216386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec85d19f5024763829a22b26a07a2df)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:07.218369 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:07.327093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:07.332529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:07.396438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cb395-bacd-4acd-8628-105f5de6dd53 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:07.401858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:07.407111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:07.407111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:07.407111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:07.407111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:07.416202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:07.417778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:07.429430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1314/2602] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:07.520765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec85d19f5024763829a22b26a07a2df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.618861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:07.620608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:07.635549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec85d19f5024763829a22b26a07a2df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.636601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec85d19f5024763829a22b26a07a2df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.639024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a8c1f-0f20-495d-9bc1-01b73e3eaef7 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:07.646261 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:07.646569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec85d19f5024763829a22b26a07a2df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.649615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:07.650145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:07.664131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] PATH_INFO: `/v3/users/200fe0a2c1814150a1ced34e35aca65a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:07.664131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] RBAC: Authorizing `identity:delete_user(user_id=200fe0a2c1814150a1ced34e35aca65a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:07.664131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:07.664503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-66cf7d46-7cd7-4063-ac56-879a24db7699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec85d19f5024763829a22b26a07a2df'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.672382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1289/2603] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:07 2022] DELETE /identity/v3/users/9ec85d19f5024763829a22b26a07a2df => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:07.676707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:07.677012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:07.678030 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:07.713113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:07.714497 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:07.790272 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1290/2604] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:07.802520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:07.805509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:07.912223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998cf8d-363d-484a-8d4e-7bcfb0407509 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:07.919176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75b432-4140-4827-9541-796fba576d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:07.919818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b432-4140-4827-9541-796fba576d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:07.920095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b432-4140-4827-9541-796fba576d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:07.920536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75b432-4140-4827-9541-796fba576d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:07.933702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75b432-4140-4827-9541-796fba576d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:07.935507 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75b432-4140-4827-9541-796fba576d57 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:07.946479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200fe0a2c1814150a1ced34e35aca65a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.947273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200fe0a2c1814150a1ced34e35aca65a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:07.952325 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1291/2605] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:07.956980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200fe0a2c1814150a1ced34e35aca65a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:08.015454 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:08.102865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200fe0a2c1814150a1ced34e35aca65a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:08.102865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200fe0a2c1814150a1ced34e35aca65a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:08.130021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:08.136390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1315/2606] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:07 2022] DELETE /identity/v3/users/200fe0a2c1814150a1ced34e35aca65a => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:08.141082 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:08.141370 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:08.141650 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:08.163297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:08.164676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:08.241626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1292/2607] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:08.250318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:08.253691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:08.342692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cdb0f8-a0a4-450f-8305-7eb202e4d20a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:08.346778 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:08.347436 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:08.347749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:08.348019 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:08.360803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:08.362245 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:08.373607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1316/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:08.606509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:08.652287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:08.653773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:08.670622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d79ac-6ac1-414d-a714-88e295282d66 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:08.676134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:08.677098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:08.680561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:08.680561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] PATH_INFO: `/v3/projects/6fb14b430d8d4d91804d791970096eec` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:08.680561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb14b430d8d4d91804d791970096eec)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:08.681074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:08.876863 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb14b430d8d4d91804d791970096eec', 'request_id': 'req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:08.979357 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:08.979992 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:09.005936 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb14b430d8d4d91804d791970096eec', 'request_id': 'req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.012130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf29002-7cff-4785-b38e-58de0f47da05 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:09.026709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb14b430d8d4d91804d791970096eec', 'request_id': 'req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.060755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] The token cache is being invalidate because project 6fb14b430d8d4d91804d791970096eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:09.064412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.064412 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.068448 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1293/2609] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:08 2022] DELETE /identity/v3/projects/6fb14b430d8d4d91804d791970096eec => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:09.069776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:09.071986 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:09.123024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:09.124591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:09.138893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:09.138893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] PATH_INFO: `/v3/projects/7e486064976041c1bc944b795cc129b5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:09.138893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e486064976041c1bc944b795cc129b5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:09.138893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:09.225959 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603216c-6a6a-4bbb-a588-b7c0bb60fccd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:09.234126 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:09.234954 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:09.235338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:09.235682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] PATH_INFO: `/v3/users/bc901045d54e463a9eeffc80190c15e5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:09.238155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc901045d54e463a9eeffc80190c15e5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:09.241695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:09.302182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:09.320005 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e486064976041c1bc944b795cc129b5', 'request_id': 'req-ce98446d-c92c-44ea-90d8-e96b6b773bcb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.328070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e486064976041c1bc944b795cc129b5', 'request_id': 'req-ce98446d-c92c-44ea-90d8-e96b6b773bcb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.458583 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e486064976041c1bc944b795cc129b5', 'request_id': 'req-ce98446d-c92c-44ea-90d8-e96b6b773bcb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.470786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] The token cache is being invalidate because project 7e486064976041c1bc944b795cc129b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:09.471064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.471797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.473420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1317/2610] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:08 2022] DELETE /identity/v3/projects/7e486064976041c1bc944b795cc129b5 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:09.478981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:09.480861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:09.582609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98446d-c92c-44ea-90d8-e96b6b773bcb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:09.588437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:09.588705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:09.589087 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:09.589371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:09.589682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] PATH_INFO: `/v3/users/7816b2da1a3b4bff92299a85fb6df2f8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:09.591054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7816b2da1a3b4bff92299a85fb6df2f8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:09.592901 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:09.637380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc901045d54e463a9eeffc80190c15e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.720633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc901045d54e463a9eeffc80190c15e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.721523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc901045d54e463a9eeffc80190c15e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.736778 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc901045d54e463a9eeffc80190c15e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.760445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc901045d54e463a9eeffc80190c15e5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.771510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1294/2611] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:09 2022] DELETE /identity/v3/users/bc901045d54e463a9eeffc80190c15e5 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:09.944813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7816b2da1a3b4bff92299a85fb6df2f8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.945479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7816b2da1a3b4bff92299a85fb6df2f8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:09.953887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7816b2da1a3b4bff92299a85fb6df2f8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.085375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7816b2da1a3b4bff92299a85fb6df2f8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.087717 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8d59edcb-df42-4e5f-8f20-7b96bc651ae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7816b2da1a3b4bff92299a85fb6df2f8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.123453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1318/2612] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:09 2022] DELETE /identity/v3/users/7816b2da1a3b4bff92299a85fb6df2f8 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:10.356098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:10.357628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:10.500788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52691e-e294-4357-a517-b9902f6ee5fc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:10.576951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:10.577759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:10.578042 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:10.578320 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] PATH_INFO: `/v3/projects/a5592966e3cc40c9b778c1e5a6194d30` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:10.579578 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5592966e3cc40c9b778c1e5a6194d30)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:10.581754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:10.779788 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5592966e3cc40c9b778c1e5a6194d30', 'request_id': 'req-1cbd8541-ebc1-45ed-b591-e16bfc665f93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.855630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5592966e3cc40c9b778c1e5a6194d30', 'request_id': 'req-1cbd8541-ebc1-45ed-b591-e16bfc665f93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.863756 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5592966e3cc40c9b778c1e5a6194d30', 'request_id': 'req-1cbd8541-ebc1-45ed-b591-e16bfc665f93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.876765 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] The token cache is being invalidate because project a5592966e3cc40c9b778c1e5a6194d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:10.877338 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.878504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:10.880191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1295/2613] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:10 2022] DELETE /identity/v3/projects/a5592966e3cc40c9b778c1e5a6194d30 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:10.886960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f97d1-3433-4d3a-94b4-1065b1e11557 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:10.888666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f97d1-3433-4d3a-94b4-1065b1e11557 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:10.888666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f97d1-3433-4d3a-94b4-1065b1e11557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:10.923787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-914f97d1-3433-4d3a-94b4-1065b1e11557 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:10.925368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-914f97d1-3433-4d3a-94b4-1065b1e11557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:10.991008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:10.993821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:11.000432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1319/2614] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:11.011498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.012249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.012975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.024728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd8541-ebc1-45ed-b591-e16bfc665f93 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.034675 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.036078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.036595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.037496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] PATH_INFO: `/v3/projects/faa3894a79a0459ab90142f4facf4e3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.041326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa3894a79a0459ab90142f4facf4e3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.046215 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:11.046408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.052508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:11.068368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1320/2615] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:11.070955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.091399 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c00d7-ed66-41c9-be5b-158b1c4b79fd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.101554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.102937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.102937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.102937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.103496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:11.103866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.105453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.120548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1321/2616] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:53:11.126764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.138249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e735e1-2ee9-46a4-bc03-796bb2dd8ccb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.141940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:11.146783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.148606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.148912 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.154069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.154069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.154069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.208767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa3894a79a0459ab90142f4facf4e3f', 'request_id': 'req-c48d1c9b-f0dd-4c73-9802-8567610f08f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:11.217843 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:11.277612 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1322/2617] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:11.281968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.301231 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa3894a79a0459ab90142f4facf4e3f', 'request_id': 'req-c48d1c9b-f0dd-4c73-9802-8567610f08f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:11.311381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa3894a79a0459ab90142f4facf4e3f', 'request_id': 'req-c48d1c9b-f0dd-4c73-9802-8567610f08f3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:11.323792 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] The token cache is being invalidate because project faa3894a79a0459ab90142f4facf4e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:11.324107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:11.325473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c48d1c9b-f0dd-4c73-9802-8567610f08f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:11.327036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1296/2618] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:10 2022] DELETE /identity/v3/projects/faa3894a79a0459ab90142f4facf4e3f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:11.335305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ba7f-206f-48fe-891b-d0b6773b27d6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.335521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ba7f-206f-48fe-891b-d0b6773b27d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.335800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ba7f-206f-48fe-891b-d0b6773b27d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.356850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-30e2ba7f-206f-48fe-891b-d0b6773b27d6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:11.357868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17caec-5d2d-4e51-a3fa-113bb56f8114 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.358068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30e2ba7f-206f-48fe-891b-d0b6773b27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:11.365172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.366330 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.366613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.366877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.367562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1297/2619] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:11.367709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.369565 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.383186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.383562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.383909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.410663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:11.412721 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:11.426276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1298/2620] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:11.427629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.438661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ed4f6-c950-427a-9389-87eeb48c04a3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.447044 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.449673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.449673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.449673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.449673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:11.449673 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.450781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.468857 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1299/2621] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:53:11.473815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.487901 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000d03-ce74-415b-b4eb-6b45d9e206d8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.499025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.499654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.499933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.500203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.500880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.503214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.510407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1323/2622] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:11.514561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.516289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:11.529646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa2c3c-7714-454c-b209-a631af2f4e77 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.535510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.536121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.544760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.544760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.544760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.544760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.553611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1324/2623] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:11.558656 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.569166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06fa2e-ccb0-4270-a5f0-4b723365a6d3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.583055 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.583704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.584047 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.584291 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.585609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.587234 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.598803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1325/2624] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:11.604956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.616065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcc14f-7214-472a-9708-25464b7d9997 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.621602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3f7d-0dda-47c3-bb36-6a321b06b91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.622265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3f7d-0dda-47c3-bb36-6a321b06b91c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.622557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3f7d-0dda-47c3-bb36-6a321b06b91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.622808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3f7d-0dda-47c3-bb36-6a321b06b91c admin admin] PATH_INFO: `/v3/projects/a28738ded1554cefbae60822cefac7b5/users/291697d75b444875b4ecadbc96366c2d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.642237 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad3f7d-0dda-47c3-bb36-6a321b06b91c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28738ded1554cefbae60822cefac7b5, user_id=291697d75b444875b4ecadbc96366c2d, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.643398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad3f7d-0dda-47c3-bb36-6a321b06b91c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.674906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1300/2625] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:11.676810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.724823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1326/2626] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:11 2022] PUT /identity/v3/projects/a28738ded1554cefbae60822cefac7b5/users/291697d75b444875b4ecadbc96366c2d/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:11.738518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.739127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.739521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.774115 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] MFA Rules not processed for user `291697d75b444875b4ecadbc96366c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:11.775646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:11.791063 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1847f9-bb0a-44b8-937a-0fa5eeafd3bf admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.863038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1327/2627] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:11.868111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:11.925259 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.926925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.927496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.928061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.930227 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.935938 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:11.980942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bbaaa-5fc0-4dfb-8c7a-dd58c1745aa7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:11.987782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:11.988526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:11.988828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:11.989101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:11.990049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:11.992073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.072406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1301/2628] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/users => generated 294 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:12.079435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.095006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30374a7-6e80-4283-a32a-3b266ddfdfdd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.103528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.104779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.105143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.105663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.106802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.109769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.118894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1328/2629] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:11 2022] POST /identity/v3/projects => generated 387 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:12.127985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1302/2630] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:12.128871 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.137049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.149100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bbd02-a324-47c4-82ce-2cdd1595815e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.223236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c0a8a-ba81-4985-bcb1-f9eb4f9b83ee admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.225396 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.226022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.226295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.226563 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.227217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.228888 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.229490 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.230331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.230765 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.231111 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.232009 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.234275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.246566 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1303/2631] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:12.247750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.263378 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2a89-9485-400a-a350-66f972cd2bec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.276060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e7ba7-8ee1-43ae-814d-ac93db8d2462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.277187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e7ba7-8ee1-43ae-814d-ac93db8d2462 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.281582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e7ba7-8ee1-43ae-814d-ac93db8d2462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.281582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e7ba7-8ee1-43ae-814d-ac93db8d2462 admin admin] PATH_INFO: `/v3/projects/31e5bbe065e548fd8cf3c32671e12fb3/users/63506f5b1b7a4fdc93367c275ff2cbce/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.301134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e7ba7-8ee1-43ae-814d-ac93db8d2462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e5bbe065e548fd8cf3c32671e12fb3, user_id=63506f5b1b7a4fdc93367c275ff2cbce, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.305092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e7ba7-8ee1-43ae-814d-ac93db8d2462 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.433424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1329/2632] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/users => generated 294 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:12.437278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.447371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094ac37-52f6-4cc6-a8a5-99ccfb8f48bf admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.452828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.453531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.453863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.454308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.455387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.457393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.468423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1330/2633] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:12.472441 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.482603 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:12.483438 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1304/2634] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:12 2022] PUT /identity/v3/projects/31e5bbe065e548fd8cf3c32671e12fb3/users/63506f5b1b7a4fdc93367c275ff2cbce/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:12.491694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89c3cb-aedd-40bb-8a1c-9f2688e89678 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.494798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.495143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.495433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.528728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] MFA Rules not processed for user `63506f5b1b7a4fdc93367c275ff2cbce`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:12.530023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:12.558961 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.559578 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.559859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.560213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.561356 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.562536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.576240 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1331/2635] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:12.580933 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.591341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:12.598523 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5e02d-ed90-4d1a-80df-c430d94f06e8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.601991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041b949-5307-423d-8c91-3714ebcd5029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.602738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041b949-5307-423d-8c91-3714ebcd5029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.603112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041b949-5307-423d-8c91-3714ebcd5029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.603429 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041b949-5307-423d-8c91-3714ebcd5029 admin admin] PATH_INFO: `/v3/projects/99a0c8d2eb07412fbe9f6ca7ba2723c1/users/267688a53e3b4dfc8bc2b7484e9859b4/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.617245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1305/2636] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:12.620125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041b949-5307-423d-8c91-3714ebcd5029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a0c8d2eb07412fbe9f6ca7ba2723c1, user_id=267688a53e3b4dfc8bc2b7484e9859b4, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.622647 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.622921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041b949-5307-423d-8c91-3714ebcd5029 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.636095 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b57f3-15bc-42d2-934c-496c61fee7fc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.641382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.642084 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.642404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.655890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.655890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.655890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.803084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1332/2637] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:12 2022] PUT /identity/v3/projects/99a0c8d2eb07412fbe9f6ca7ba2723c1/users/267688a53e3b4dfc8bc2b7484e9859b4/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:12.805339 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.805584 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1306/2638] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:12.805714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.805917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.809317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.839002 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] MFA Rules not processed for user `267688a53e3b4dfc8bc2b7484e9859b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:12.844897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:12.890472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583a08b-89d1-406f-9b13-ba1506dc58eb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.898094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.898834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.899031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.899273 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.899912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.901931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:12.939733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1333/2639] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:12.944496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:12.955996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab46532-7a16-4e12-b2dc-aba96c0d299c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:12.963051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:12.963662 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:12.964011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:12.964290 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:12.964991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:12.966861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.067181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1307/2640] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/users => generated 299 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:13.072753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.082523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e44f17-3295-4761-bf48-2d86e7c47dd8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.088193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.088838 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.089114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.089380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.090048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.091667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.107381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1308/2641] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:13.109591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1334/2642] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:12 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:13.110256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.113208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.120655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2f7b3-3175-4973-85e8-44b4befea684 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.200914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.202249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.202595 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.202973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] PATH_INFO: `/v3/projects/43c8cab1fc0f46b883767360780a0539/users/0436fc3d376b42ae9e7663d822d8555d/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.204617 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c2138-63d3-405a-923f-bd3d0aa919a1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.213705 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.214699 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.215231 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.215700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.216820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.219691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.227444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c8cab1fc0f46b883767360780a0539, user_id=0436fc3d376b42ae9e7663d822d8555d, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.229443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.264497 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:13.345244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1309/2643] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:13 2022] PUT /identity/v3/projects/43c8cab1fc0f46b883767360780a0539/users/0436fc3d376b42ae9e7663d822d8555d/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:13.349392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.375584 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1335/2644] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:13 2022] POST /identity/v3/users => generated 300 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:13.379622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.399879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2952250-2827-451e-8630-92ddc02df997 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.467158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433bbdb-57ca-4dd6-8888-a3db575e376d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.474686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.475785 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.475935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.476125 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.477065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.484426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.496029 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1310/2645] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:13.497571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.513717 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b12cc-b82d-4118-802b-6014487089f5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.540469 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff1d8c-fe85-4149-857f-d4415ddf8ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.540751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff1d8c-fe85-4149-857f-d4415ddf8ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.541039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff1d8c-fe85-4149-857f-d4415ddf8ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.541784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff1d8c-fe85-4149-857f-d4415ddf8ce2 admin admin] PATH_INFO: `/v3/projects/43c8cab1fc0f46b883767360780a0539/users/0436fc3d376b42ae9e7663d822d8555d/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.541885 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.542993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.543423 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.544536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.545041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff1d8c-fe85-4149-857f-d4415ddf8ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c8cab1fc0f46b883767360780a0539, user_id=0436fc3d376b42ae9e7663d822d8555d, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.547308 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.554971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff1d8c-fe85-4149-857f-d4415ddf8ce2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.561693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.576183 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1336/2647] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:13.581540 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.586920 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1311/2647] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:13 2022] PUT /identity/v3/projects/43c8cab1fc0f46b883767360780a0539/users/0436fc3d376b42ae9e7663d822d8555d/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:13.589397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.589694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.590429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.612968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11753ce7-81c1-4dff-9f62-a00c45873b8e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.632503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] MFA Rules not processed for user `0436fc3d376b42ae9e7663d822d8555d`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:13.634582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:13.708862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.709763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.710351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.710704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] PATH_INFO: `/v3/projects/403e5151753e4f6898c7037cf1ee1d18/users/05c96763c7084301baa7aebfbc4d157e/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.743363 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403e5151753e4f6898c7037cf1ee1d18, user_id=05c96763c7084301baa7aebfbc4d157e, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.747931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.754255 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1312/2648] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:13.774810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.776539 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:13.833033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1337/2649] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:13 2022] PUT /identity/v3/projects/403e5151753e4f6898c7037cf1ee1d18/users/05c96763c7084301baa7aebfbc4d157e/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:13.837082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:13.901384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7160145-53f0-464e-b4c2-9969f1f820c9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.942103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:13.958967 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7e085-ab1b-4e98-83ea-d41b08e6e155 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:13.970252 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:13.970911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:13.971194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:13.971474 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:13.972181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:13.974448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:13.988073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1338/2650] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:13.996904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.012093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd06b2-1c18-4f2d-9e51-7def2ec1b29d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.019132 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36c669-135e-43ca-917d-245d0e15e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.019816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.019816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.020244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.020244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.020744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36c669-135e-43ca-917d-245d0e15e6f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.026153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36c669-135e-43ca-917d-245d0e15e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.026153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36c669-135e-43ca-917d-245d0e15e6f3 admin admin] PATH_INFO: `/v3/projects/403e5151753e4f6898c7037cf1ee1d18/users/05c96763c7084301baa7aebfbc4d157e/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.026153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36c669-135e-43ca-917d-245d0e15e6f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403e5151753e4f6898c7037cf1ee1d18, user_id=05c96763c7084301baa7aebfbc4d157e, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.032660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36c669-135e-43ca-917d-245d0e15e6f3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.066519 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1339/2651] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:13 2022] PUT /identity/v3/projects/403e5151753e4f6898c7037cf1ee1d18/users/05c96763c7084301baa7aebfbc4d157e/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:14.071082 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.071630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.073427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.116169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] MFA Rules not processed for user `05c96763c7084301baa7aebfbc4d157e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:14.118545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.128799 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.130275 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.144101 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1313/2652] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:14.182971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.200486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc3d5f-b1fb-4f05-bac2-d178e504836f service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.228671 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1340/2653] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:14.236597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:14.239129 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:14.297688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.298425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.301425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.301425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] PATH_INFO: `/v3/users/63506f5b1b7a4fdc93367c275ff2cbce` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.301425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63506f5b1b7a4fdc93367c275ff2cbce)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.307775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.331704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e23d09-3a0a-46aa-8357-a2ea35a10cb1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:14.339248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18e31e-79b5-4790-815e-b263638437bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:14.339248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18e31e-79b5-4790-815e-b263638437bd service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.339248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18e31e-79b5-4790-815e-b263638437bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.339647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18e31e-79b5-4790-815e-b263638437bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.363445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:14.369944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18e31e-79b5-4790-815e-b263638437bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:14.369944 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18e31e-79b5-4790-815e-b263638437bd service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:14.385595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1341/2654] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:14.691755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63506f5b1b7a4fdc93367c275ff2cbce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:14.843831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63506f5b1b7a4fdc93367c275ff2cbce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:14.844524 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63506f5b1b7a4fdc93367c275ff2cbce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:14.868494 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63506f5b1b7a4fdc93367c275ff2cbce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:14.891825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63506f5b1b7a4fdc93367c275ff2cbce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:14.900013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1314/2655] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:14 2022] DELETE /identity/v3/users/63506f5b1b7a4fdc93367c275ff2cbce => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:14.905110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:14.905591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:14.905964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:14.937369 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:14.939186 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:15.028028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1342/2656] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:15.040263 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:15.042439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:15.122636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574d87-f167-4ba8-9719-24b6fa0de9b0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:15.127350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:15.128055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:15.128324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:15.128613 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:15.139457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:15.140967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:15.156508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1315/2657] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:15.671250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:15.673210 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:15.690526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:15.697257 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283c0a9-8952-4a0f-ad27-362ed208e6e8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:15.706477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:15.707269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:15.707919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:15.708127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] PATH_INFO: `/v3/projects/31e5bbe065e548fd8cf3c32671e12fb3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:15.710834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e5bbe065e548fd8cf3c32671e12fb3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:15.712888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:15.809880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:15.925295 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e5bbe065e548fd8cf3c32671e12fb3', 'request_id': 'req-898cab1d-82e8-4bb3-a6f5-10447de68d64'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:15.933861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e5bbe065e548fd8cf3c32671e12fb3', 'request_id': 'req-898cab1d-82e8-4bb3-a6f5-10447de68d64'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.005203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:16.007656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:16.055979 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e5bbe065e548fd8cf3c32671e12fb3', 'request_id': 'req-898cab1d-82e8-4bb3-a6f5-10447de68d64'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.073732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] The token cache is being invalidate because project 31e5bbe065e548fd8cf3c32671e12fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:16.074323 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.075658 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.079119 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1343/2658] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:15 2022] DELETE /identity/v3/projects/31e5bbe065e548fd8cf3c32671e12fb3 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:16.083341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:16.085749 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:16.098102 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69b3a7-39f8-48cb-a3f1-3bee711e6e4d service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:16.179674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:16.181217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:16.181746 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:16.182262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:16.185908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:16.216608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cab1d-82e8-4bb3-a6f5-10447de68d64 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:16.227343 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:16.228397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:16.228397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:16.228397 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] PATH_INFO: `/v3/users/0436fc3d376b42ae9e7663d822d8555d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:16.231130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0436fc3d376b42ae9e7663d822d8555d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:16.233219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:16.296825 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:16.298394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:16.309213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1316/2659] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:16.508355 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0436fc3d376b42ae9e7663d822d8555d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.509836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0436fc3d376b42ae9e7663d822d8555d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.522611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0436fc3d376b42ae9e7663d822d8555d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.671927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0436fc3d376b42ae9e7663d822d8555d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.673457 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1a8f8f6d-9962-4405-80d6-f524ddc69d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0436fc3d376b42ae9e7663d822d8555d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:16.693201 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1344/2660] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:16 2022] DELETE /identity/v3/users/0436fc3d376b42ae9e7663d822d8555d => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:17.026526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:17.091946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:17.177244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:17.191035 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dcff69-3b7b-4f78-915e-30e1a9df3f84 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:17.289113 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:17.290088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:17.290088 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:17.290522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] PATH_INFO: `/v3/projects/43c8cab1fc0f46b883767360780a0539` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:17.292162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c8cab1fc0f46b883767360780a0539)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:17.294527 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:17.478883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c8cab1fc0f46b883767360780a0539', 'request_id': 'req-e7fa8890-b966-4b5f-88e2-7adac144bc24'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:17.609365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c8cab1fc0f46b883767360780a0539', 'request_id': 'req-e7fa8890-b966-4b5f-88e2-7adac144bc24'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:17.617583 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c8cab1fc0f46b883767360780a0539', 'request_id': 'req-e7fa8890-b966-4b5f-88e2-7adac144bc24'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:17.633831 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] The token cache is being invalidate because project 43c8cab1fc0f46b883767360780a0539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:17.634345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:17.635277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7fa8890-b966-4b5f-88e2-7adac144bc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:17.636861 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1317/2661] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:17 2022] DELETE /identity/v3/projects/43c8cab1fc0f46b883767360780a0539 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:17.642424 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:17.643508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:17.643508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:17.666437 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:17.667650 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:17.749392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1345/2662] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:17.757955 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:17.758420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:17.758945 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:17.782337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:17.783780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:17.795860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1318/2663] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:17.800651 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:17.811410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b22a1-d161-4374-9915-97a36ba997ea None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:17.817085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:17.817775 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:17.818054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:17.818389 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:17.819173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:17.819458 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:17.821174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:17.832968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1346/2664] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:53:17.837759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:17.847468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242c2bf-005f-48e7-917b-84d1dd1c4c3a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:17.853162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:17.853779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:17.854055 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:17.854322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:17.854947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:17.857014 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.001543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1319/2665] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:17 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:18.005693 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:18.017798 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7203c0-0502-41d1-8f48-4ce093780b0b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:18.080272 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:18.080916 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.081195 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.081473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.082081 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:18.083996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.277508 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1347/2666] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:18 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:18.281655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:18.293480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b4705-3f42-4b84-b3bc-cbecc2b3661d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:18.298656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:18.299346 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.299625 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.300077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.300940 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:18.302797 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.317965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1320/2667] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:18.319485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:18.330841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de16bb2-5020-4275-bad7-36fef5ca4209 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:18.338939 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d193833-d326-4659-a98f-75bc180f24db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:18.340168 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d193833-d326-4659-a98f-75bc180f24db admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.340806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d193833-d326-4659-a98f-75bc180f24db admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.341346 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d193833-d326-4659-a98f-75bc180f24db admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.342665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d193833-d326-4659-a98f-75bc180f24db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:18.345813 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d193833-d326-4659-a98f-75bc180f24db admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.365660 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1348/2668] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:18.374479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:18.388274 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13959-a760-4b95-aa98-54b0b190a30b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:18.393640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:18.394464 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.394846 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.395229 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] PATH_INFO: `/v3/projects/2e192d883d054171a4c80001bf9997d6/users/ec06a233344e412fbd1a76265df6e4b8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.418592 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e192d883d054171a4c80001bf9997d6, user_id=ec06a233344e412fbd1a76265df6e4b8, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:18.421817 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.569384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1321/2669] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:18 2022] PUT /identity/v3/projects/2e192d883d054171a4c80001bf9997d6/users/ec06a233344e412fbd1a76265df6e4b8/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:18.572083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.572409 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.573241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.595225 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:18.609514 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] MFA Rules not processed for user `ec06a233344e412fbd1a76265df6e4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:18.611064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:18.700266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1349/2670] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:18.704860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:18.717157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:18.810956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b9412-03ba-49f2-93fd-dae0b08e0cba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:18.816443 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:18.821295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:18.821295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:18.821295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:18.821295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:18.821295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:18.979122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1322/2671] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:18 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:18.984994 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:19.011851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bfa3d-ba44-4534-98cf-96aa9ac5a436 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:19.105312 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:19.106027 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:19.106318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:19.106613 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:19.107317 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:19.109596 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:19.208897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1350/2672] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:18 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:19.209534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:19.223949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054ef1-9c13-4b49-8c02-8dda63ba5e70 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:19.230505 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:19.231691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:19.232066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:19.232452 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:19.233394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:19.235011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:19.248473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1323/2673] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:19.252686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:19.265628 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721856c-d819-47cb-b44b-9352560a68a7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:19.270858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c001-dfa4-4f43-a947-251eda90ae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:19.271473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c001-dfa4-4f43-a947-251eda90ae09 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:19.271810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c001-dfa4-4f43-a947-251eda90ae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:19.272857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c001-dfa4-4f43-a947-251eda90ae09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:19.273653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33c001-dfa4-4f43-a947-251eda90ae09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:19.275290 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33c001-dfa4-4f43-a947-251eda90ae09 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:19.286221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1351/2674] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:19.291172 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:19.304679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfc76f-77f2-438f-b324-80555d8d2dca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:19.309472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:19.309971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:19.310390 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:19.310660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] PATH_INFO: `/v3/projects/ec7b4a83a70243e9a0176f674d763fc5/users/8cbe3112e9e94843b2b26bd21956886a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:19.465146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7b4a83a70243e9a0176f674d763fc5, user_id=8cbe3112e9e94843b2b26bd21956886a, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:19.467898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:19.626100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1324/2675] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:19 2022] PUT /identity/v3/projects/ec7b4a83a70243e9a0176f674d763fc5/users/8cbe3112e9e94843b2b26bd21956886a/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:19.638414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:19.638692 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:19.639037 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:19.667797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] MFA Rules not processed for user `8cbe3112e9e94843b2b26bd21956886a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:19.673987 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:19.756527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1352/2676] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:19.762700 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:19.864300 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89bee0-50a9-46f7-aca3-7b229c5142eb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:19.873411 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:19.874423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:19.874804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:19.875167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:19.876097 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:19.884127 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:19.983403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1325/2677] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:19 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:19.988130 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:20.013869 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f75f9-729e-48c9-9855-cbb19b44b0e7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:20.087948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:20.088608 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.089067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.089318 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.089956 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:20.091934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:20.265131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1353/2678] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:19 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:20.269138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:20.280860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841c542-db9c-46f1-a263-ddd53951657d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:20.285723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:20.286349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.286619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.286887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.288149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:20.289845 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:20.303317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1326/2679] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:20.307800 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:20.318669 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfec5f-790b-4f28-ad6b-a318a55d003e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:20.323857 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:20.324526 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.324827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.325278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] PATH_INFO: `/v3/projects/4e547bb383aa4a9d88cfba4198a75f1a/users/86b64852a2ab4cac9056781a57c8897a/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.353823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e547bb383aa4a9d88cfba4198a75f1a, user_id=86b64852a2ab4cac9056781a57c8897a, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:20.357101 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:20.480218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1354/2680] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:20 2022] PUT /identity/v3/projects/4e547bb383aa4a9d88cfba4198a75f1a/users/86b64852a2ab4cac9056781a57c8897a/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:20.484725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:20.495772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78169ffa-6c32-4831-8a23-703d971f5175 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:20.562349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2df003-aea3-4b95-a579-844e14839bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:20.562968 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2df003-aea3-4b95-a579-844e14839bed admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.563241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2df003-aea3-4b95-a579-844e14839bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.563499 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2df003-aea3-4b95-a579-844e14839bed admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.564143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2df003-aea3-4b95-a579-844e14839bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:20.565787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2df003-aea3-4b95-a579-844e14839bed admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:20.580376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1327/2681] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:20.594683 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:20.598980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c03cb6-7602-4a25-9d44-d0cb3437d1f2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:20.605724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:20.606609 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.606888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.607144 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] PATH_INFO: `/v3/projects/4e547bb383aa4a9d88cfba4198a75f1a/users/86b64852a2ab4cac9056781a57c8897a/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.609867 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e547bb383aa4a9d88cfba4198a75f1a, user_id=86b64852a2ab4cac9056781a57c8897a, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:20.612587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:20.635332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1355/2682] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:20 2022] PUT /identity/v3/projects/4e547bb383aa4a9d88cfba4198a75f1a/users/86b64852a2ab4cac9056781a57c8897a/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:20.638162 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.638540 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.638913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.671641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] MFA Rules not processed for user `86b64852a2ab4cac9056781a57c8897a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:20.672937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:20.754719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1328/2683] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:20.768003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:20.769927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:20.771372 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:20.837990 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c27d0-59b9-45f1-832f-63bb8b618914 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:20.841810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aec2f9-6add-4369-9b20-b4f9738dba9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:20.842421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec2f9-6add-4369-9b20-b4f9738dba9d service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:20.842698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec2f9-6add-4369-9b20-b4f9738dba9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:20.842968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aec2f9-6add-4369-9b20-b4f9738dba9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:20.852864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec2f9-6add-4369-9b20-b4f9738dba9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:20.857744 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aec2f9-6add-4369-9b20-b4f9738dba9d service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:20.864889 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1356/2684] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:21.067974 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:21.070290 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:21.085289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583ccb-89b9-454d-abbd-638a343ab8e5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:21.090152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:21.091018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:21.091607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:21.092258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:21.138182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:21.138702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:21.138829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:21.162351 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] MFA Rules not processed for user `651140ed5a704cde978be56393e0537e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:21.164689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:21.185146 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:21.186729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:21.197732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1329/2685] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:21.237616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1357/2686] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:21.769841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:22.199123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:22.209887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3e459-0975-4f7f-9a13-ac603fa95534 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:22.213993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:22.214730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:22.215069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:22.215414 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:22.217487 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:22.301114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:22.302904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13368e17-4b84-49e0-aec3-e0d732c25b8f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:22.325278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1330/2687] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:24.339329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:24.467196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:25.416689 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:26.544866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:26.634056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bba52-95b8-4400-8bc3-c4e80181a88a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:26.635949 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:26.636637 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:26.636913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:26.637181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] PATH_INFO: `/v3/users/ec06a233344e412fbd1a76265df6e4b8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:26.638342 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec06a233344e412fbd1a76265df6e4b8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:26.640910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:26.958966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:26.967359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec06a233344e412fbd1a76265df6e4b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:26.968125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec06a233344e412fbd1a76265df6e4b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:26.986302 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec06a233344e412fbd1a76265df6e4b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:27.077946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec06a233344e412fbd1a76265df6e4b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:27.078907 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec06a233344e412fbd1a76265df6e4b8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:27.101610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1358/2688] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:26 2022] DELETE /identity/v3/users/ec06a233344e412fbd1a76265df6e4b8 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:27.106420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:27.107068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:27.107068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:27.127072 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:27.128297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:27.222105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1331/2689] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:27.232620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:27.234258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:27.310394 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095cdb84-647d-4a5c-844e-6062007863a5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:27.325839 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:27.326075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:27.326856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:27.329573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:27.356229 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:27.360468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:27.385071 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1359/2690] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:27.994450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:27.999576 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:28.023012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d491f5e-f19e-4053-84f2-df9e7304eb97 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:28.030816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:28.032176 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:28.032973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:28.033379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] PATH_INFO: `/v3/projects/2e192d883d054171a4c80001bf9997d6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:28.035248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e192d883d054171a4c80001bf9997d6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:28.038616 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:28.248053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e192d883d054171a4c80001bf9997d6', 'request_id': 'req-44193980-0f4a-423b-9624-fe55b0624228'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:28.409326 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e192d883d054171a4c80001bf9997d6', 'request_id': 'req-44193980-0f4a-423b-9624-fe55b0624228'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:28.420135 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e192d883d054171a4c80001bf9997d6', 'request_id': 'req-44193980-0f4a-423b-9624-fe55b0624228'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:28.435061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] The token cache is being invalidate because project 2e192d883d054171a4c80001bf9997d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:28.435356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:28.436186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:28.437735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1332/2691] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:27 2022] DELETE /identity/v3/projects/2e192d883d054171a4c80001bf9997d6 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:28.441734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:28.454534 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adafd6-a5f3-4867-9623-ccfc3ee9dc54 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:28.519976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:28.520232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:28.520446 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:28.520719 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] PATH_INFO: `/v3/users/8cbe3112e9e94843b2b26bd21956886a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:28.522724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbe3112e9e94843b2b26bd21956886a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:28.524274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:28.622234 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:28.623804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:28.637840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44193980-0f4a-423b-9624-fe55b0624228 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:28.642759 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:28.643405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:28.643678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:28.643950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] PATH_INFO: `/v3/users/291697d75b444875b4ecadbc96366c2d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:28.645174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] RBAC: Authorizing `identity:delete_user(user_id=291697d75b444875b4ecadbc96366c2d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:28.647165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:28.926603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:29.014443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbe3112e9e94843b2b26bd21956886a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.015636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbe3112e9e94843b2b26bd21956886a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.028585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbe3112e9e94843b2b26bd21956886a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.044874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291697d75b444875b4ecadbc96366c2d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.161073 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbe3112e9e94843b2b26bd21956886a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.161872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbe3112e9e94843b2b26bd21956886a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.189937 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1360/2692] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:28 2022] DELETE /identity/v3/users/8cbe3112e9e94843b2b26bd21956886a => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:29.192059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:29.196239 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:29.228525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291697d75b444875b4ecadbc96366c2d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.229983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291697d75b444875b4ecadbc96366c2d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.247056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291697d75b444875b4ecadbc96366c2d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.270948 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0443b94b-0e3e-4c2e-8c90-5b06565f677f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291697d75b444875b4ecadbc96366c2d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:29.281870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1333/2693] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:28 2022] DELETE /identity/v3/users/291697d75b444875b4ecadbc96366c2d => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:29.286511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:29.287045 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:29.287802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:29.314087 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:29.315381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:29.341787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f90b-4bba-42a9-bc3a-cd02a80da3cb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:29.385483 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1334/2694] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:29.396005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:29.398291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:29.416601 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:29.417557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:29.418046 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:29.418427 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:29.420448 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:29.496100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7c02e-9b4b-43e8-8024-78f475da9410 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:29.502098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:29.503111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:29.503602 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:29.504403 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:29.520849 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:29.524191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:29.530105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:29.531684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:29.543713 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1361/2695] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:29.544897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1335/2696] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:29.837309 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:29.839213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:29.865372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15b68-9a4d-48b7-a53f-b726073ce7c7 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:29.872634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:29.873485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:29.873862 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:29.874282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] PATH_INFO: `/v3/projects/ec7b4a83a70243e9a0176f674d763fc5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:29.875864 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7b4a83a70243e9a0176f674d763fc5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:29.878543 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:30.117720 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7b4a83a70243e9a0176f674d763fc5', 'request_id': 'req-e0ef7030-8324-40da-85b4-c77e7bc2d565'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.125790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7b4a83a70243e9a0176f674d763fc5', 'request_id': 'req-e0ef7030-8324-40da-85b4-c77e7bc2d565'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.292483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7b4a83a70243e9a0176f674d763fc5', 'request_id': 'req-e0ef7030-8324-40da-85b4-c77e7bc2d565'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.309882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] The token cache is being invalidate because project ec7b4a83a70243e9a0176f674d763fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:30.310265 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.310940 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.312480 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1362/2697] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:29 2022] DELETE /identity/v3/projects/ec7b4a83a70243e9a0176f674d763fc5 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:30.316528 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:30.328106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a18273-b16c-4aec-ac60-c3cdb08fe013 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:30.385912 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:30.386445 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:30.386723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:30.386993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] PATH_INFO: `/v3/users/86b64852a2ab4cac9056781a57c8897a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:30.388239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b64852a2ab4cac9056781a57c8897a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:30.390211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:30.439041 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:30.440686 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:30.457402 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef7030-8324-40da-85b4-c77e7bc2d565 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:30.489341 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:30.492525 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:30.492810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:30.493084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] PATH_INFO: `/v3/projects/a28738ded1554cefbae60822cefac7b5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:30.494385 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28738ded1554cefbae60822cefac7b5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:30.497454 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:30.519412 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:30.945800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b64852a2ab4cac9056781a57c8897a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.953284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28738ded1554cefbae60822cefac7b5', 'request_id': 'req-5cd9f572-fd75-473d-b8ca-baf06f23929f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:30.955297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28738ded1554cefbae60822cefac7b5', 'request_id': 'req-5cd9f572-fd75-473d-b8ca-baf06f23929f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.118015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b64852a2ab4cac9056781a57c8897a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.118806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b64852a2ab4cac9056781a57c8897a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.127001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b64852a2ab4cac9056781a57c8897a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.141509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b64852a2ab4cac9056781a57c8897a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.148556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1336/2698] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:30 2022] DELETE /identity/v3/users/86b64852a2ab4cac9056781a57c8897a => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:31.224975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28738ded1554cefbae60822cefac7b5', 'request_id': 'req-5cd9f572-fd75-473d-b8ca-baf06f23929f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.241247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] The token cache is being invalidate because project a28738ded1554cefbae60822cefac7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:31.241726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.242482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:31.246941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1363/2699] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:30 2022] DELETE /identity/v3/projects/a28738ded1554cefbae60822cefac7b5 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:31.250628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:31.252858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:31.360057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c9546-3611-4780-b368-8f40e115a136 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:31.366629 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:31.367308 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:31.376998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:31.376998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] PATH_INFO: `/v3/users/267688a53e3b4dfc8bc2b7484e9859b4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:31.376998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=267688a53e3b4dfc8bc2b7484e9859b4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:31.376998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:31.641779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:31.859590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '267688a53e3b4dfc8bc2b7484e9859b4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.029331 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '267688a53e3b4dfc8bc2b7484e9859b4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.034421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '267688a53e3b4dfc8bc2b7484e9859b4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.038103 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '267688a53e3b4dfc8bc2b7484e9859b4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.052880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '267688a53e3b4dfc8bc2b7484e9859b4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.064937 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1337/2700] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:31 2022] DELETE /identity/v3/users/267688a53e3b4dfc8bc2b7484e9859b4 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:32.329877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:32.331258 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:32.443828 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9f572-fd75-473d-b8ca-baf06f23929f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:32.565276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:32.565925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:32.566203 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:32.566477 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] PATH_INFO: `/v3/projects/4e547bb383aa4a9d88cfba4198a75f1a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:32.567623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e547bb383aa4a9d88cfba4198a75f1a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:32.573335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:32.767976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:32.769939 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:32.797213 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f73e5-0a2b-4f90-b1ec-90eeef727920 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:32.810697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e547bb383aa4a9d88cfba4198a75f1a', 'request_id': 'req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.826741 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e547bb383aa4a9d88cfba4198a75f1a', 'request_id': 'req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.929474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:32.930332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:32.930573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:32.931051 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] PATH_INFO: `/v3/projects/99a0c8d2eb07412fbe9f6ca7ba2723c1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:32.933905 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a0c8d2eb07412fbe9f6ca7ba2723c1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:32.935858 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:32.938782 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e547bb383aa4a9d88cfba4198a75f1a', 'request_id': 'req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.972890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] The token cache is being invalidate because project 4e547bb383aa4a9d88cfba4198a75f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:32.973410 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.980062 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a69dbdeb-cddb-446f-ac90-79e3ec255e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:32.982506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1364/2701] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:32 2022] DELETE /identity/v3/projects/4e547bb383aa4a9d88cfba4198a75f1a => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:32.989214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06092-a6c9-45b5-bfb6-057f4268d76e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:32.991179 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06092-a6c9-45b5-bfb6-057f4268d76e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:32.991708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06092-a6c9-45b5-bfb6-057f4268d76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.032602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ada06092-a6c9-45b5-bfb6-057f4268d76e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:33.034110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ada06092-a6c9-45b5-bfb6-057f4268d76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:33.060438 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1365/2702] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:33.066670 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:33.104366 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a0c8d2eb07412fbe9f6ca7ba2723c1', 'request_id': 'req-9a59894d-57e3-47f3-a34c-41cc08e9211a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.123723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.124016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.124276 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.157821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:33.159220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:33.199138 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a0c8d2eb07412fbe9f6ca7ba2723c1', 'request_id': 'req-9a59894d-57e3-47f3-a34c-41cc08e9211a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.209568 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a0c8d2eb07412fbe9f6ca7ba2723c1', 'request_id': 'req-9a59894d-57e3-47f3-a34c-41cc08e9211a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.240960 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] The token cache is being invalidate because project 99a0c8d2eb07412fbe9f6ca7ba2723c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:33.246638 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.248723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.249978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1366/2703] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:33.250643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1338/2704] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:32 2022] DELETE /identity/v3/projects/99a0c8d2eb07412fbe9f6ca7ba2723c1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:33.254863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.255143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.257054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:33.257196 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:33.271242 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3450c64-c5d3-4f5d-b326-67af77129126 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:33.279053 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:33.290011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.290011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.290011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] PATH_INFO: `/v3/users/05c96763c7084301baa7aebfbc4d157e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.290011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c96763c7084301baa7aebfbc4d157e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:33.290011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:33.297039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59894d-57e3-47f3-a34c-41cc08e9211a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:33.310988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:33.311927 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.313010 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.313480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.314489 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:33.314975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:33.323479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:33.342786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1339/2705] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:53:33.347554 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.365286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71faec-beac-403a-8cc3-9d44600e4c66 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:33.376764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:33.378658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.379354 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.379837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.380706 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:33.384691 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:33.513043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1340/2706] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:33 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:33.518913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.589383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c96763c7084301baa7aebfbc4d157e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.590774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c96763c7084301baa7aebfbc4d157e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.605931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c96763c7084301baa7aebfbc4d157e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.645567 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c5fd0-64ea-4289-9c28-3388bdbc0fe1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:33.658165 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:33.659049 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.660033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.660433 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.661181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:33.664230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:33.750434 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c96763c7084301baa7aebfbc4d157e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.751075 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c96763c7084301baa7aebfbc4d157e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:33.759455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1341/2707] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:33 2022] POST /identity/v3/users => generated 284 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:33.761410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.763181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:33.782213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1367/2708] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:33 2022] DELETE /identity/v3/users/05c96763c7084301baa7aebfbc4d157e => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:33.782429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e771-5d5a-416c-94b9-f59167cf8b29 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:33.892549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:33.893851 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.894342 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.895077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.897886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:33.904393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:33.927818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1342/2709] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:33.932763 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.950748 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214597c-ded9-4e54-9bdd-1239bafbe493 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:33.958911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413ac1b-bbcc-462f-bb39-ed744e1247fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:33.960881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413ac1b-bbcc-462f-bb39-ed744e1247fd admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:33.960881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413ac1b-bbcc-462f-bb39-ed744e1247fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:33.961188 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413ac1b-bbcc-462f-bb39-ed744e1247fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:33.961911 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413ac1b-bbcc-462f-bb39-ed744e1247fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:33.963668 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413ac1b-bbcc-462f-bb39-ed744e1247fd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:33.980442 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1368/2710] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:33.981069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:33.998546 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ef823-293f-474e-a218-0c8d735d551e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.016543 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.018075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.018796 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.019217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] PATH_INFO: `/v3/projects/45ab8605fcd042dda56300ea487c36c5/users/ece4474dd85141a3934f3ede6c233b5e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.047699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ab8605fcd042dda56300ea487c36c5, user_id=ece4474dd85141a3934f3ede6c233b5e, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.052860 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:34.198516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1343/2711] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:33 2022] PUT /identity/v3/projects/45ab8605fcd042dda56300ea487c36c5/users/ece4474dd85141a3934f3ede6c233b5e/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:34.209836 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.210156 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.210392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.264069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] MFA Rules not processed for user `ece4474dd85141a3934f3ede6c233b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:34.265921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:34.352707 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1369/2712] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:34.357805 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:34.425071 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea88ab6-4299-4c71-997b-24ccb4624b72 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.430597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.431198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.431474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.431741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.432387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.434299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:34.484017 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:34.485997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:34.505331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9859e-7edd-45c8-97aa-9abf0cb799c0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.517485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.517485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.517485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.517485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] PATH_INFO: `/v3/projects/403e5151753e4f6898c7037cf1ee1d18` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.518466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=403e5151753e4f6898c7037cf1ee1d18)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.518466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:34.581815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1344/2713] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:34 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:34.586735 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:34.664398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd722d-731c-48a5-991d-7d49e2c5d58d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.669705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.670327 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.672593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.672593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.672593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.673575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:34.689267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403e5151753e4f6898c7037cf1ee1d18', 'request_id': 'req-562139ae-d8bc-4c58-93a5-e1440eae6fc2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:34.701236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403e5151753e4f6898c7037cf1ee1d18', 'request_id': 'req-562139ae-d8bc-4c58-93a5-e1440eae6fc2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:34.810618 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1345/2714] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:34 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:34.814667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:34.825512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284428-1326-4b10-b9d7-ea78feb05081 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.842056 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:34.842981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403e5151753e4f6898c7037cf1ee1d18', 'request_id': 'req-562139ae-d8bc-4c58-93a5-e1440eae6fc2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:34.859825 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] The token cache is being invalidate because project 403e5151753e4f6898c7037cf1ee1d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:34.860665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:34.861646 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-562139ae-d8bc-4c58-93a5-e1440eae6fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:34.863993 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1370/2715] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:34 2022] DELETE /identity/v3/projects/403e5151753e4f6898c7037cf1ee1d18 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:34.869524 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476597f5-c8ad-4d81-ac65-3f46311c1ef6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.869897 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476597f5-c8ad-4d81-ac65-3f46311c1ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.870226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476597f5-c8ad-4d81-ac65-3f46311c1ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.891826 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.892481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.892597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-476597f5-c8ad-4d81-ac65-3f46311c1ef6 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:34.892880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.893019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.893680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.893808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-476597f5-c8ad-4d81-ac65-3f46311c1ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:34.895482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:34.903530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1371/2716] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:34.907008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1346/2717] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:34.912623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.912913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.915873 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.916005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:34.916005 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:34.929316 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4963f1d-fba6-4b78-a2d3-a5b763d68cc0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.937633 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.939093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.939233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:34.939764 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.940440 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] PATH_INFO: `/v3/projects/f6680cfcc2f54e3f8092fdc14000856b/users/f1cd3e23284d45378186048e4f135cc1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.940687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:34.951283 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1372/2718] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:34.959057 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:34.959386 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6680cfcc2f54e3f8092fdc14000856b, user_id=f1cd3e23284d45378186048e4f135cc1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.962306 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:34.972059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbf543-9d45-4e66-b6a0-73ad8e5f2c29 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:34.983694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:34.990172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:34.990172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:34.990172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:34.990172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:34.990172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:34.990172 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.001856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1373/2719] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:53:35.006590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.017502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd734535-6909-49e9-9bfc-ab94c8f82713 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.023827 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.024632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.031501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.031501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.031501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.031501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.067776 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1347/2720] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:34 2022] PUT /identity/v3/projects/f6680cfcc2f54e3f8092fdc14000856b/users/f1cd3e23284d45378186048e4f135cc1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:35.076158 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.145993 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3098b26-7b50-4d74-9ab1-9997d8a77c9f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.151747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.152388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.152667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.152933 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.153577 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.155874 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.167347 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1348/2721] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:35.171909 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.176248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1374/2722] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:35 2022] POST /identity/v3/projects => generated 391 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:35.183213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.183630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957f815-88ce-4e0e-b770-938c4dc21ebd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.262170 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd85b1-263d-4b4a-832f-58329d649ae2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.263116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b60a9-7a6b-42af-879f-6bad7d1ea440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.263784 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b60a9-7a6b-42af-879f-6bad7d1ea440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.264056 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b60a9-7a6b-42af-879f-6bad7d1ea440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.264324 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b60a9-7a6b-42af-879f-6bad7d1ea440 admin admin] PATH_INFO: `/v3/projects/f6680cfcc2f54e3f8092fdc14000856b/users/f1cd3e23284d45378186048e4f135cc1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.266358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b60a9-7a6b-42af-879f-6bad7d1ea440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6680cfcc2f54e3f8092fdc14000856b, user_id=f1cd3e23284d45378186048e4f135cc1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.268759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.269073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b60a9-7a6b-42af-879f-6bad7d1ea440 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.269751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.270228 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.270689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.271793 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.275159 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.286291 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1349/2723] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:35 2022] PUT /identity/v3/projects/f6680cfcc2f54e3f8092fdc14000856b/users/f1cd3e23284d45378186048e4f135cc1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:35.298148 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.298435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.298818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.339435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] MFA Rules not processed for user `f1cd3e23284d45378186048e4f135cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:35.340724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:35.417704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1375/2724] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:35 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:35.417704 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.428651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1350/2725] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:35.430594 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62530b-e72a-4f3d-a7b5-3ac6e5435c80 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.447823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.451066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:35.536698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f972-6a38-4050-a583-10715233a501 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.543751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.543943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.544127 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.544244 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.544366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.544747 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.544851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.544976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.545768 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.547580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.558949 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.567267 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1376/2726] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:35.567628 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.568495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.574623 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1351/2727] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:35.583563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45ec0-c8df-4f04-be77-3b6e112a7fbc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.589638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069a17d-228d-4d6e-affb-b2eda5b399eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.590600 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a17d-228d-4d6e-affb-b2eda5b399eb admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.590882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a17d-228d-4d6e-affb-b2eda5b399eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.591166 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a17d-228d-4d6e-affb-b2eda5b399eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.591966 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069a17d-228d-4d6e-affb-b2eda5b399eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.593746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069a17d-228d-4d6e-affb-b2eda5b399eb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.609049 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1377/2728] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:35.610281 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.623513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33979fb0-ca38-46ae-b323-0b36db3f91ca service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.628671 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.629404 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.629684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.630036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] PATH_INFO: `/v3/projects/6cd6541ffb0c4e4ea186bee5bc42872a/users/751d1d987eac4bfa8983284b59adb3e1/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.651534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd6541ffb0c4e4ea186bee5bc42872a, user_id=751d1d987eac4bfa8983284b59adb3e1, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.654364 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:35.776573 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1352/2729] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:35 2022] PUT /identity/v3/projects/6cd6541ffb0c4e4ea186bee5bc42872a/users/751d1d987eac4bfa8983284b59adb3e1/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:35.788831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.789414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.789414 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.821140 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] MFA Rules not processed for user `751d1d987eac4bfa8983284b59adb3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:35.822373 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:35.867587 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.896236 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1378/2730] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:35.901991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:35.931070 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43f9b5-32cc-4f69-8273-5b85abcc8508 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.935557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.936183 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.936490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.943280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.943280 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:35.975517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fac3e-061f-42f1-8898-abcbfddb1caf None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:35.980606 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:35.981216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:35.981496 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:35.981766 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:35.982425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:35.984483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.006761 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:36.008408 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.017967 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1353/2731] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:36.104095 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1379/2732] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:35 2022] POST /identity/v3/projects => generated 391 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:36.108394 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:36.123388 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a70d1-ba3d-4739-8a5f-f3878e14bd6c service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:36.191731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:36.192365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:36.192790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:36.193075 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:36.194932 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:36.196956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.299304 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1354/2733] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:36 2022] POST /identity/v3/users => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:36.303794 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:36.323968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215152a-2fdc-4d22-bfc0-f25789763610 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:36.331136 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:36.332475 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:36.332716 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:36.333143 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:36.334653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:36.336435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.347315 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1380/2734] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:36.352897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:36.364295 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d138e-ea3b-4675-b019-5d1021de4783 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:36.370462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:36.370462 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:36.370871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:36.370871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] PATH_INFO: `/v3/projects/b1bb4270f839477f9779710f41d0bc1c/users/824c43e466ac49cbbebab6def7e13e7d/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:36.388419 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bb4270f839477f9779710f41d0bc1c, user_id=824c43e466ac49cbbebab6def7e13e7d, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:36.391278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.537652 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1355/2735] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:36 2022] PUT /identity/v3/projects/b1bb4270f839477f9779710f41d0bc1c/users/824c43e466ac49cbbebab6def7e13e7d/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:36.542416 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:36.553687 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd900f-a07e-4944-9586-20119bb82024 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:36.624235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b2d60-33c3-414d-a6b3-c8dce36a4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:36.625378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b2d60-33c3-414d-a6b3-c8dce36a4c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:36.625786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b2d60-33c3-414d-a6b3-c8dce36a4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:36.626120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b2d60-33c3-414d-a6b3-c8dce36a4c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:36.629094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b2d60-33c3-414d-a6b3-c8dce36a4c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:36.630917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b2d60-33c3-414d-a6b3-c8dce36a4c20 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.661493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1381/2736] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:36.663614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:36.686786 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ffa27-4305-4bdf-921f-48870bebed5a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:36.694239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:36.695266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:36.695728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:36.696128 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] PATH_INFO: `/v3/projects/b1bb4270f839477f9779710f41d0bc1c/users/824c43e466ac49cbbebab6def7e13e7d/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:36.699018 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bb4270f839477f9779710f41d0bc1c, user_id=824c43e466ac49cbbebab6def7e13e7d, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:36.703803 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:36.725422 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1356/2737] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:36 2022] PUT /identity/v3/projects/b1bb4270f839477f9779710f41d0bc1c/users/824c43e466ac49cbbebab6def7e13e7d/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:36.737616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:36.737900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:36.738177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:36.775244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] MFA Rules not processed for user `824c43e466ac49cbbebab6def7e13e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:36.776529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:36.876795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1382/2738] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:36.973694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:37.050678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b4aa4-c128-4b72-8c11-f79fd5bfc4f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:37.055247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:37.055895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:37.056164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:37.056465 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:37.067287 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:37.069512 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:37.083198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1357/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:36 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:37.185893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:37.198162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe2d92-f674-48c4-a7d9-0f912139db90 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:37.202372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cda590-d844-4e11-ab38-de1f4fecbf86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:37.209792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda590-d844-4e11-ab38-de1f4fecbf86 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:37.209792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda590-d844-4e11-ab38-de1f4fecbf86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:37.209792 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cda590-d844-4e11-ab38-de1f4fecbf86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:37.243789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:37.275573 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cda590-d844-4e11-ab38-de1f4fecbf86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:37.277099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cda590-d844-4e11-ab38-de1f4fecbf86 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:37.287063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1383/2740] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:37.644404 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:40.479887 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:41.019100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:42.962067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:43.264990 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:43.336798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23e303-bac7-447b-a361-9ea2a4cfc4d3 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:43.341542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:43.342174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:43.342453 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:43.350688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] PATH_INFO: `/v3/users/751d1d987eac4bfa8983284b59adb3e1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:43.350688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=751d1d987eac4bfa8983284b59adb3e1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:43.350688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:43.351026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:43.589601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751d1d987eac4bfa8983284b59adb3e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:43.746446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751d1d987eac4bfa8983284b59adb3e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:43.747220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751d1d987eac4bfa8983284b59adb3e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:43.755740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751d1d987eac4bfa8983284b59adb3e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:43.771078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751d1d987eac4bfa8983284b59adb3e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:43.781941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1358/2741] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:43 2022] DELETE /identity/v3/users/751d1d987eac4bfa8983284b59adb3e1 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:43.783590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:43.783868 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:43.784334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:43.805980 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:43.807251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:43.869077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1384/2742] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:43.881753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:43.881753 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:43.949124 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71553c-887a-43b6-9da9-9c66bf3ae373 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:43.953425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:43.954109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:43.954455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:43.954795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:43.970173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:43.972117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:43.983885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1359/2743] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:44.116278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:44.117855 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:44.129891 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec6692-6fbc-44e7-a65f-58b87915abe2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:44.135905 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:44.136358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:44.136653 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:44.136927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] PATH_INFO: `/v3/users/ece4474dd85141a3934f3ede6c233b5e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:44.138069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece4474dd85141a3934f3ede6c233b5e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:44.139986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:44.426903 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece4474dd85141a3934f3ede6c233b5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.427580 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece4474dd85141a3934f3ede6c233b5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.435232 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece4474dd85141a3934f3ede6c233b5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.557391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:44.559709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:44.577351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9f68-7498-4153-a5bd-202241c2a2ee service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:44.582392 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:44.583942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:44.585048 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:44.585450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] PATH_INFO: `/v3/projects/6cd6541ffb0c4e4ea186bee5bc42872a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:44.586812 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd6541ffb0c4e4ea186bee5bc42872a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:44.588884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:44.615642 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece4474dd85141a3934f3ede6c233b5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.616403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7b2fb2e6-06a7-44d1-a4d5-e0800e0d668e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece4474dd85141a3934f3ede6c233b5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.644100 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1385/2744] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:44 2022] DELETE /identity/v3/users/ece4474dd85141a3934f3ede6c233b5e => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:44.647975 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:44.648558 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:44.649024 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:44.670830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:44.672250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:44.754900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1386/2745] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:44.773684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:44.775125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:44.826062 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd6541ffb0c4e4ea186bee5bc42872a', 'request_id': 'req-70b7afff-2614-43cf-a808-e675060272e3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.839219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce0c12-5531-4e03-af3c-acfba7863338 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:44.850703 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:44.905712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:44.906878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:44.907298 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:44.907638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:44.975373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd6541ffb0c4e4ea186bee5bc42872a', 'request_id': 'req-70b7afff-2614-43cf-a808-e675060272e3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.979829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:44.981419 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:44.983177 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd6541ffb0c4e4ea186bee5bc42872a', 'request_id': 'req-70b7afff-2614-43cf-a808-e675060272e3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.993069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1387/2746] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:44.997317 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] The token cache is being invalidate because project 6cd6541ffb0c4e4ea186bee5bc42872a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:44.997707 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:44.998476 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:45.000264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1360/2747] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:44 2022] DELETE /identity/v3/projects/6cd6541ffb0c4e4ea186bee5bc42872a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:45.004220 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:45.005976 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:45.020638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4db06-9e12-42d0-b2b2-0d3d45d4f246 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:45.030207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:45.030865 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:45.031221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:45.036411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] PATH_INFO: `/v3/users/824c43e466ac49cbbebab6def7e13e7d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:45.036411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=824c43e466ac49cbbebab6def7e13e7d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:45.036411 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:45.489266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824c43e466ac49cbbebab6def7e13e7d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:45.490089 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824c43e466ac49cbbebab6def7e13e7d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:45.500602 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824c43e466ac49cbbebab6def7e13e7d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:45.602182 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:45.723521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824c43e466ac49cbbebab6def7e13e7d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:45.724067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824c43e466ac49cbbebab6def7e13e7d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:45.745176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1388/2748] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:45 2022] DELETE /identity/v3/users/824c43e466ac49cbbebab6def7e13e7d => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:45.749977 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:45.752381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:45.776249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7afff-2614-43cf-a808-e675060272e3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:45.867729 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:45.868421 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:45.868702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:45.868973 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] PATH_INFO: `/v3/projects/45ab8605fcd042dda56300ea487c36c5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:45.870210 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ab8605fcd042dda56300ea487c36c5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:45.872237 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:46.032755 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ab8605fcd042dda56300ea487c36c5', 'request_id': 'req-8b045abe-9157-4925-9e37-fe7cbaf1eea8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.161379 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ab8605fcd042dda56300ea487c36c5', 'request_id': 'req-8b045abe-9157-4925-9e37-fe7cbaf1eea8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.171208 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ab8605fcd042dda56300ea487c36c5', 'request_id': 'req-8b045abe-9157-4925-9e37-fe7cbaf1eea8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.188682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] The token cache is being invalidate because project 45ab8605fcd042dda56300ea487c36c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:46.189001 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.189680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.191267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1361/2749] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:45 2022] DELETE /identity/v3/projects/45ab8605fcd042dda56300ea487c36c5 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:46.197033 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:46.199110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:46.213567 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e01397-2bc9-488f-923b-f4678ef28a61 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:46.249111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:46.249223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:46.293107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:46.293755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:46.294026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:46.294296 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] PATH_INFO: `/v3/users/f1cd3e23284d45378186048e4f135cc1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:46.295780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1cd3e23284d45378186048e4f135cc1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:46.297780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:46.301152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:46.326536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b045abe-9157-4925-9e37-fe7cbaf1eea8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:46.331789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:46.332457 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:46.332732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:46.332998 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] PATH_INFO: `/v3/projects/b1bb4270f839477f9779710f41d0bc1c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:46.334152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bb4270f839477f9779710f41d0bc1c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:46.336249 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:46.586093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1cd3e23284d45378186048e4f135cc1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.586725 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1cd3e23284d45378186048e4f135cc1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.589898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bb4270f839477f9779710f41d0bc1c', 'request_id': 'req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.594760 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1cd3e23284d45378186048e4f135cc1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.686620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bb4270f839477f9779710f41d0bc1c', 'request_id': 'req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.696630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bb4270f839477f9779710f41d0bc1c', 'request_id': 'req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.711474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] The token cache is being invalidate because project b1bb4270f839477f9779710f41d0bc1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:46.711975 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.712980 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3ddfc839-96c1-4daa-bb44-e25a88d10bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.714551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1362/2750] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:46 2022] DELETE /identity/v3/projects/b1bb4270f839477f9779710f41d0bc1c => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:46.719794 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:46.720107 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:46.720374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:46.752074 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:46.753385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:46.796157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1cd3e23284d45378186048e4f135cc1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.796930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-00136e7b-f24b-4bc1-b619-1891b89ff122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1cd3e23284d45378186048e4f135cc1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:46.831173 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1389/2751] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:46 2022] DELETE /identity/v3/users/f1cd3e23284d45378186048e4f135cc1 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:46.840060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1363/2752] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:46.853727 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:46.854611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:46.855623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:46.891384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:46.892691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:46.902109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:46.985849 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1390/2753] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:46.991129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.001450 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0e108-23e5-4570-9a0f-e20f35edf6a3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.007590 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.008219 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.008534 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.008885 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.009551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:47.009884 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.011605 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.028581 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1364/2754] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:53:47.033238 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.047856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db73730-c28c-46ef-b594-b973da4fdc6f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.050887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.051528 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.051812 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.052094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.054320 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.056383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.173808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1391/2755] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:47.177908 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.189740 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53465dee-2678-4ba1-a207-c81612cee69f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.272258 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.272903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.273187 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.273458 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.274112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.276332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.319185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:47.383838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.386069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:47.406691 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74841d79-5105-4b94-aa44-6203bfe82247 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.408942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1365/2756] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:47.413426 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.417431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.417586 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.418176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.418921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] PATH_INFO: `/v3/projects/f6680cfcc2f54e3f8092fdc14000856b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.421311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6680cfcc2f54e3f8092fdc14000856b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.424926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132940ad-d1fb-4a8b-b8f2-e1587cd76d70 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.425175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.437008 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.437617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.437891 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.438455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.439061 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.440752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.456332 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1366/2757] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:47.457777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.468943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea4283-14bb-47a1-9338-bfb2fbbfbcea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.473728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.474365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.474688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.474966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.476009 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.477732 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.488199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1367/2758] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:47.493470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.503790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15177d79-d791-4108-9ddb-77531d80cec3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.510537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057ef57-b118-43c2-8b83-dac8e3fe669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.511222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057ef57-b118-43c2-8b83-dac8e3fe669d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.511496 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057ef57-b118-43c2-8b83-dac8e3fe669d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.511767 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057ef57-b118-43c2-8b83-dac8e3fe669d admin admin] PATH_INFO: `/v3/projects/a48e4ea686a54c57975efd1e7005ba51/users/a2e3378592da4abc914193b8e74f693f/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.531848 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057ef57-b118-43c2-8b83-dac8e3fe669d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48e4ea686a54c57975efd1e7005ba51, user_id=a2e3378592da4abc914193b8e74f693f, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.534582 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057ef57-b118-43c2-8b83-dac8e3fe669d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.619946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1368/2759] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:47 2022] PUT /identity/v3/projects/a48e4ea686a54c57975efd1e7005ba51/users/a2e3378592da4abc914193b8e74f693f/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:47.629592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6680cfcc2f54e3f8092fdc14000856b', 'request_id': 'req-c0dc066a-60e0-4c39-8431-d93afae83030'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:47.632091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.632285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.632475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.638077 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6680cfcc2f54e3f8092fdc14000856b', 'request_id': 'req-c0dc066a-60e0-4c39-8431-d93afae83030'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:47.655048 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:47.665532 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] MFA Rules not processed for user `a2e3378592da4abc914193b8e74f693f`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:47.666791 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:47.703853 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6680cfcc2f54e3f8092fdc14000856b', 'request_id': 'req-c0dc066a-60e0-4c39-8431-d93afae83030'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:47.718968 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] The token cache is being invalidate because project f6680cfcc2f54e3f8092fdc14000856b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:47.719305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:47.719925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c0dc066a-60e0-4c39-8431-d93afae83030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:47.721506 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1392/2760] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:47 2022] DELETE /identity/v3/projects/f6680cfcc2f54e3f8092fdc14000856b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:47.729694 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b636-bc7e-4cf8-9e12-0d9167c7d8e0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.730038 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b636-bc7e-4cf8-9e12-0d9167c7d8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.730405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b636-bc7e-4cf8-9e12-0d9167c7d8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.753160 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1369/2761] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:47.758498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e624b636-bc7e-4cf8-9e12-0d9167c7d8e0 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:47.758498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e624b636-bc7e-4cf8-9e12-0d9167c7d8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:47.761787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.763536 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:47.849535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66337a-ec2f-4832-86df-620a9b29d10e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.855079 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1393/2762] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:47.861329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.862241 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.862521 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.862806 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.863471 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.863711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.863982 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.864536 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.866841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.878853 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:47.903028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:47.903028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:47.911301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1394/2763] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:47.916654 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.929374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790c4dd-18a4-4236-acb4-a69ae48e19c3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.933926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.934562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.937162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.937162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.937162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:47.937162 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.937702 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.950463 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1395/2764] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:53:47.955590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:47.966941 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8d645-68ad-40b3-91f4-4e1bb0441dd2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:47.972190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:47.972900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:47.973181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:47.973453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:47.974147 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:47.976737 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:47.988266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1370/2765] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:47.996500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.068381 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752490ac-340f-4431-8cb3-dc2e4b8509c3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.073834 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.074472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.074751 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.075019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.075690 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.077737 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.114269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1396/2766] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:48.118531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.192950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1371/2767] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:47 2022] POST /identity/v3/users => generated 298 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:48.198509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.208244 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c88b0f-2d76-4a5c-844a-c5122a6e0ca5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.214442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17342b-2c57-4d62-abbb-b098474257fd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.217632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.218563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.218986 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.219405 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.220546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.221674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.226382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.226382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.226382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.226382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.226899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.226987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.238015 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1372/2768] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:48.242663 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.253978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca91536-8d81-45c9-9d8e-8cb01e55a4f4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.258757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:48.264657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.266947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.266947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.266947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] PATH_INFO: `/v3/projects/49d86ff517d5432ca8503b690a57c75a/users/9525eac007a748a8ae8f851eb86c21f1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.288053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d86ff517d5432ca8503b690a57c75a, user_id=9525eac007a748a8ae8f851eb86c21f1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.290748 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.425681 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1397/2769] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:48 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:48.430406 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.440872 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0f63b-c147-47d3-af2b-82755cca80d0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.447587 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.448208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.448557 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.448834 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.449450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.451065 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.462432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1398/2770] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:48.468368 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.480909 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76418e99-d95c-4987-a5e8-d4a1e308e7f2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.488334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.489604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.490213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.491761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.492214 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.495069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.507698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1399/2771] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:48.512020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.522810 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe7845-8351-4349-aefe-47898e7647b8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.531065 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1373/2772] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:48 2022] PUT /identity/v3/projects/49d86ff517d5432ca8503b690a57c75a/users/9525eac007a748a8ae8f851eb86c21f1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:48.543762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.600538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd15c8-f206-46cf-b701-d8d3c81a57a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.601233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd15c8-f206-46cf-b701-d8d3c81a57a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.601509 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd15c8-f206-46cf-b701-d8d3c81a57a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.601780 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd15c8-f206-46cf-b701-d8d3c81a57a3 admin admin] PATH_INFO: `/v3/projects/199068f3a41640dfac446d942712791c/users/b4e81017a6d249e5a557cf1ac3580d15/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.615840 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:48.624493 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd15c8-f206-46cf-b701-d8d3c81a57a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199068f3a41640dfac446d942712791c, user_id=b4e81017a6d249e5a557cf1ac3580d15, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.628590 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd15c8-f206-46cf-b701-d8d3c81a57a3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.641651 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a22be79-8297-4748-9d8e-5aa642252e6c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.647635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.648276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.648620 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.648946 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.649615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.651349 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.652305 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:48.667122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1374/2773] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:48.672279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.684674 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62a740-c631-4471-ba79-8d8ccec9c179 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.689611 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4e4de-3d19-43a1-b2c4-772acf0219a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.690264 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4e4de-3d19-43a1-b2c4-772acf0219a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.690617 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4e4de-3d19-43a1-b2c4-772acf0219a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.690889 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4e4de-3d19-43a1-b2c4-772acf0219a6 admin admin] PATH_INFO: `/v3/projects/49d86ff517d5432ca8503b690a57c75a/users/9525eac007a748a8ae8f851eb86c21f1/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.692850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4e4de-3d19-43a1-b2c4-772acf0219a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d86ff517d5432ca8503b690a57c75a, user_id=9525eac007a748a8ae8f851eb86c21f1, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.695657 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4e4de-3d19-43a1-b2c4-772acf0219a6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:48.706867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1375/2774] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:48 2022] PUT /identity/v3/projects/49d86ff517d5432ca8503b690a57c75a/users/9525eac007a748a8ae8f851eb86c21f1/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:48.728964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1400/2775] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:48 2022] PUT /identity/v3/projects/199068f3a41640dfac446d942712791c/users/b4e81017a6d249e5a557cf1ac3580d15/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:48.742036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.742036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.742036 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.780401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] MFA Rules not processed for user `b4e81017a6d249e5a557cf1ac3580d15`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:48.781191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.781661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.781800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:48.782180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.822803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] MFA Rules not processed for user `9525eac007a748a8ae8f851eb86c21f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:48.824114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:48.874695 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1376/2776] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:48.879516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.926360 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1401/2777] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:48.938425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:48.939779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:48.973589 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f12c-dd00-4cff-b761-848c97c55286 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:48.982925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3da34d-2045-458b-a1f2-aa5c3bb2176b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:48.983934 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3da34d-2045-458b-a1f2-aa5c3bb2176b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:48.984655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3da34d-2045-458b-a1f2-aa5c3bb2176b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:48.985003 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3da34d-2045-458b-a1f2-aa5c3bb2176b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:48.986100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3da34d-2045-458b-a1f2-aa5c3bb2176b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:48.989736 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3da34d-2045-458b-a1f2-aa5c3bb2176b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.056152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:49.060866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe47213-e8d6-4e45-8825-f77a63c54ae8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.064851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.065472 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.065752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.066043 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.128674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.130175 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.141723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1402/2778] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:49.183522 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1377/2779] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:48 2022] POST /identity/v3/projects => generated 387 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:53:49.187983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:49.191026 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:49.211332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd520-f8fc-407b-b277-61f6abe42b59 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.220481 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.220996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.220996 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.256877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:49.258515 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:49.284697 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.285393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.285840 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.286114 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.286751 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.289935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.346805 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1378/2780] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Aug 24 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:49.413856 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1403/2781] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:49 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:49.417997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:49.430248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c58460-799c-4ed5-bf4e-f3fd98688d60 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.435356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.436041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.436365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.436850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.437793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.439477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.457588 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1379/2782] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:49.461564 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:49.480107 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031eb4c7-cc2a-4a50-ae62-2852ab16c196 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.487191 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.488059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.488378 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.488647 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] PATH_INFO: `/v3/projects/36b19e45240b4cbb9116bf83d6d5da64/users/9e6d925dddc743e1897343f19e316255/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.508003 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] RBAC: Authorizing `identity:create_grant(project_id=36b19e45240b4cbb9116bf83d6d5da64, user_id=9e6d925dddc743e1897343f19e316255, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.510761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.526862 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:49.529586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:49.586116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1404/2783] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:49 2022] PUT /identity/v3/projects/36b19e45240b4cbb9116bf83d6d5da64/users/9e6d925dddc743e1897343f19e316255/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:49.591659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:49.592633 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:49.635033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc4d9c-1458-4c33-85cc-09e58f86c582 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.680597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e4e4f-a1e3-4b97-b154-61ac2438954e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.687935 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.688890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.689218 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.689595 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.690769 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.692797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.709621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1405/2784] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:53:49.709895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.709895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.709895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.709895 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.710597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:49.721374 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f01d1f-7675-4371-b804-55fc27dfff4a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:49.727093 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaf9ec-4729-4e99-81fe-2e7661f70f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:49.727821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9ec-4729-4e99-81fe-2e7661f70f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.728165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9ec-4729-4e99-81fe-2e7661f70f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.728396 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaf9ec-4729-4e99-81fe-2e7661f70f4c admin admin] PATH_INFO: `/v3/projects/36b19e45240b4cbb9116bf83d6d5da64/users/9e6d925dddc743e1897343f19e316255/roles/20693bab78b1461298baee2d4c8e1f27` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.730433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9ec-4729-4e99-81fe-2e7661f70f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=36b19e45240b4cbb9116bf83d6d5da64, user_id=9e6d925dddc743e1897343f19e316255, role_id=20693bab78b1461298baee2d4c8e1f27)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.733199 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaf9ec-4729-4e99-81fe-2e7661f70f4c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.747180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1406/2785] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:49 2022] PUT /identity/v3/projects/36b19e45240b4cbb9116bf83d6d5da64/users/9e6d925dddc743e1897343f19e316255/roles/20693bab78b1461298baee2d4c8e1f27 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:53:49.759757 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:49.760045 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:49.760334 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:49.795930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:49.798158 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] MFA Rules not processed for user `9e6d925dddc743e1897343f19e316255`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:49.799629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:49.801698 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:49.804429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:49.817942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1380/2786] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:53:49.956401 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1407/2787] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:53:49.971039 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:50.093449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:50.105053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554ec6b-6a75-43e9-852b-3164392b9903 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:50.215031 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9611a-72b8-43c7-9227-2eab019c2219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:50.217068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9611a-72b8-43c7-9227-2eab019c2219 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:50.217068 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9611a-72b8-43c7-9227-2eab019c2219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:50.217557 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9611a-72b8-43c7-9227-2eab019c2219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:50.235057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9611a-72b8-43c7-9227-2eab019c2219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:50.236655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9611a-72b8-43c7-9227-2eab019c2219 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:50.253808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1381/2788] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:53:50.255384 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:50.604140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:51.020461 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:52.336563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:52.431689 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:52.481505 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859aaad-0c8d-4dd4-9708-c42516edf7fa None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:52.489791 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:52.491284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:52.491284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:52.492271 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] PATH_INFO: `/v3/users/b4e81017a6d249e5a557cf1ac3580d15` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:52.494247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e81017a6d249e5a557cf1ac3580d15)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:52.498221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:52.763078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e81017a6d249e5a557cf1ac3580d15'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:52.765544 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e81017a6d249e5a557cf1ac3580d15'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:52.779745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e81017a6d249e5a557cf1ac3580d15'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:52.885091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e81017a6d249e5a557cf1ac3580d15'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:52.888145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e81017a6d249e5a557cf1ac3580d15'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:52.937483 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1408/2789] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:52 2022] DELETE /identity/v3/users/b4e81017a6d249e5a557cf1ac3580d15 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:52.940294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:52.940832 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:52.941266 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:52.970470 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:52.971725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:53.076697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1382/2790] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:53.089118 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:53.090604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:53.174223 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d395d0b-ad05-4d5f-9fb7-678abce28bca admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:53.179745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:53.180421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:53.180701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:53.180972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:53.192213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:53.194326 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:53.205879 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1409/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:53.939966 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:53.956903 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:53.962111 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d81d10-28d5-4f12-9b43-5ce977f01132 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:53.973038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:53.974117 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:53.975297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:53.975850 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] PATH_INFO: `/v3/projects/199068f3a41640dfac446d942712791c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:53.977956 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199068f3a41640dfac446d942712791c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:53.983080 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:54.178456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:54.204926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199068f3a41640dfac446d942712791c', 'request_id': 'req-8ebfe114-e219-4ab4-9990-2ebf28e13a45'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.302415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199068f3a41640dfac446d942712791c', 'request_id': 'req-8ebfe114-e219-4ab4-9990-2ebf28e13a45'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.328574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199068f3a41640dfac446d942712791c', 'request_id': 'req-8ebfe114-e219-4ab4-9990-2ebf28e13a45'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.362299 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] The token cache is being invalidate because project 199068f3a41640dfac446d942712791c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:54.362622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.363345 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.365950 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1383/2792] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:53 2022] DELETE /identity/v3/projects/199068f3a41640dfac446d942712791c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:54.371453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:54.393035 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd290f66-afa8-4e71-b5f3-e4397a891015 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:54.517604 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:54.518745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:54.519531 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:54.520217 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] PATH_INFO: `/v3/users/9e6d925dddc743e1897343f19e316255` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:54.522332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6d925dddc743e1897343f19e316255)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:54.524623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:54.529964 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:54.909028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6d925dddc743e1897343f19e316255'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.910479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6d925dddc743e1897343f19e316255'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:54.927461 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6d925dddc743e1897343f19e316255'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:55.062922 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6d925dddc743e1897343f19e316255'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:55.063899 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6d925dddc743e1897343f19e316255'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:55.092469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1410/2793] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:53:54 2022] DELETE /identity/v3/users/9e6d925dddc743e1897343f19e316255 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:55.766787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:55.769480 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:55.837422 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:55.841128 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:55.938328 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfe114-e219-4ab4-9990-2ebf28e13a45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:55.998878 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193887f-7b63-4d95-8161-e24f110f4b0d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:56.011366 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:56.012771 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:56.013274 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:56.013772 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] PATH_INFO: `/v3/projects/36b19e45240b4cbb9116bf83d6d5da64` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:56.015759 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36b19e45240b4cbb9116bf83d6d5da64)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:56.017831 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:56.044816 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:56.045715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:56.045989 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:56.046260 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:56.048184 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:56.159040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36b19e45240b4cbb9116bf83d6d5da64', 'request_id': 'req-471cd980-59f9-4d06-b2ab-70e3c239bbe1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:56.174123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36b19e45240b4cbb9116bf83d6d5da64', 'request_id': 'req-471cd980-59f9-4d06-b2ab-70e3c239bbe1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:56.215193 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:56.230793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6644d-6900-48a7-a59c-18b3ffa4d90b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:56.268631 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1384/2794] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:53:56.278165 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36b19e45240b4cbb9116bf83d6d5da64', 'request_id': 'req-471cd980-59f9-4d06-b2ab-70e3c239bbe1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:56.305113 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] The token cache is being invalidate because project 36b19e45240b4cbb9116bf83d6d5da64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:53:56.306817 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:56.311517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-471cd980-59f9-4d06-b2ab-70e3c239bbe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:53:56.315946 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1411/2795] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:53:55 2022] DELETE /identity/v3/projects/36b19e45240b4cbb9116bf83d6d5da64 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:56.325123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:56.325614 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:56.326139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:56.366482 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:56.368712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:56.421914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:56.548559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1385/2796] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:56.582266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:56.582266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:56.582266 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:56.586377 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:56.588105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:56.597449 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1412/2797] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:56.602655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:56.616137 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34a788-8f03-4556-8aba-50a6dab42fd2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:56.622686 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:56.623322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:56.623601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:56.623871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:56.624622 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:53:56.626725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:56.628728 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:56.663091 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1386/2798] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 16:53:56.669560 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:56.679861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:53:56.684452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083da91-c115-4aac-8c1a-ab36ffc16cce None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:56.690176 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:56.691149 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:56.691485 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:56.691758 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:56.692715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:56.696611 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:57.050413 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1413/2799] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:53:56 2022] POST /identity/v3/projects => generated 389 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:57.056725 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:57.077337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438271b9-24c6-4a32-a46e-dcb16b41cf48 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:57.144823 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:57.145425 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:57.145694 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:57.145962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:57.146762 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:57.148710 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:57.356615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1387/2800] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:53:57 2022] POST /identity/v3/users => generated 295 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:53:57.358226 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:57.386372 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d886517-efec-4cba-9835-fb98e2ad9e8e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:57.398028 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:57.398799 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:57.399145 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:57.399421 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:57.400109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:57.401764 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:57.414383 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1414/2801] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:57.419389 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:57.452942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111abf1-e01f-46c5-8905-3bc3ddb07d96 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:57.461552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c83d7-0236-4002-964f-cf0f6dcf23dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:57.462374 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c83d7-0236-4002-964f-cf0f6dcf23dc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:57.462798 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c83d7-0236-4002-964f-cf0f6dcf23dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:57.463270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c83d7-0236-4002-964f-cf0f6dcf23dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:57.464243 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c83d7-0236-4002-964f-cf0f6dcf23dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:57.466719 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c83d7-0236-4002-964f-cf0f6dcf23dc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:57.487766 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1388/2802] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:53:57.493712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:57.513552 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b81aaf-565b-426d-a51c-6d578d3679e1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:57.522541 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:57.530328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:57.530328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:57.530328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] PATH_INFO: `/v3/projects/07c197e1b21c4020bdd55f84526fceb3/users/d2018d33c5434a41b2db48b714ab20c0/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:57.583607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c197e1b21c4020bdd55f84526fceb3, user_id=d2018d33c5434a41b2db48b714ab20c0, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:57.588665 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:57.709316 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1415/2803] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:53:57 2022] PUT /identity/v3/projects/07c197e1b21c4020bdd55f84526fceb3/users/d2018d33c5434a41b2db48b714ab20c0/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:53:57.727305 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:57.730775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:57.730775 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:57.809586 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] MFA Rules not processed for user `d2018d33c5434a41b2db48b714ab20c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:53:57.811923 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:57.946872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1389/2804] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:53:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:53:57.964063 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:53:57.966676 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:53:58.083645 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733f6cf-7c74-41c3-8404-c617de459f73 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:53:58.089928 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f0de1-a4f2-4530-b403-fa68257b2076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:53:58.090592 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f0de1-a4f2-4530-b403-fa68257b2076 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:53:58.090870 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f0de1-a4f2-4530-b403-fa68257b2076 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:53:58.091154 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f0de1-a4f2-4530-b403-fa68257b2076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:53:58.104255 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f0de1-a4f2-4530-b403-fa68257b2076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:53:58.105955 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f0de1-a4f2-4530-b403-fa68257b2076 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:53:58.119468 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1416/2805] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:54:05.715773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:05.720684 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:05.775598 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:05.842415 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d5716-9a41-4eef-b6d1-e1e2f9900537 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:05.851373 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:05.855073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:05.855073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:05.855073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] PATH_INFO: `/v3/users/a2e3378592da4abc914193b8e74f693f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:05.855073 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e3378592da4abc914193b8e74f693f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:05.857167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:06.235925 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e3378592da4abc914193b8e74f693f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:06.414190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e3378592da4abc914193b8e74f693f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:06.417402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e3378592da4abc914193b8e74f693f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:06.427871 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e3378592da4abc914193b8e74f693f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:06.447093 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e3378592da4abc914193b8e74f693f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:06.456969 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1390/2806] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:05 2022] DELETE /identity/v3/users/a2e3378592da4abc914193b8e74f693f => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:06.463481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:06.470723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:06.470723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:06.495327 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:06.496682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:06.612386 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1417/2807] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:54:06.628626 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:06.630734 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:06.717149 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eec52-5e72-44a6-97e6-a8e6ca58b3f3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:06.723727 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:06.725351 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:06.725828 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:06.726309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:06.742356 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:06.744180 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:06.755387 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1391/2808] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:07.478381 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:07.481926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:07.510995 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb1e8b-53af-4f9e-bf48-db700214c989 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:07.520914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:07.522103 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:07.522435 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:07.532020 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] PATH_INFO: `/v3/projects/a48e4ea686a54c57975efd1e7005ba51` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:07.537734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48e4ea686a54c57975efd1e7005ba51)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:07.541040 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:07.713443 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:07.722167 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48e4ea686a54c57975efd1e7005ba51', 'request_id': 'req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:07.731222 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48e4ea686a54c57975efd1e7005ba51', 'request_id': 'req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:07.823850 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48e4ea686a54c57975efd1e7005ba51', 'request_id': 'req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:07.844545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] The token cache is being invalidate because project a48e4ea686a54c57975efd1e7005ba51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:07.845207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:07.846187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:07.848926 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1418/2809] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:07 2022] DELETE /identity/v3/projects/a48e4ea686a54c57975efd1e7005ba51 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:07.856040 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:07.869239 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfc4d0-2a8c-4fd5-aafe-35e480f381bb service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:07.976294 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:07.999525 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:08.000230 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:08.000517 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:08.000789 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] PATH_INFO: `/v3/users/9525eac007a748a8ae8f851eb86c21f1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:08.001962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9525eac007a748a8ae8f851eb86c21f1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:08.003913 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:08.330357 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9525eac007a748a8ae8f851eb86c21f1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:08.453804 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9525eac007a748a8ae8f851eb86c21f1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:08.454815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9525eac007a748a8ae8f851eb86c21f1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:08.467442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9525eac007a748a8ae8f851eb86c21f1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:08.495155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-30d6f5c9-7127-4f34-9283-e664c1115ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9525eac007a748a8ae8f851eb86c21f1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:08.504181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1392/2810] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:07 2022] DELETE /identity/v3/users/9525eac007a748a8ae8f851eb86c21f1 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:54:09.244235 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:09.246714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:09.297886 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ae251-37d8-4882-b052-8fcf80cd4dc9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:09.398795 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:09.399918 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:09.400439 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:09.400931 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] PATH_INFO: `/v3/projects/49d86ff517d5432ca8503b690a57c75a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:09.402482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d86ff517d5432ca8503b690a57c75a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:09.405358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:09.576094 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d86ff517d5432ca8503b690a57c75a', 'request_id': 'req-19c423f1-0ed1-4778-91e5-ec2f74044003'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:09.585806 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d86ff517d5432ca8503b690a57c75a', 'request_id': 'req-19c423f1-0ed1-4778-91e5-ec2f74044003'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:09.717851 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d86ff517d5432ca8503b690a57c75a', 'request_id': 'req-19c423f1-0ed1-4778-91e5-ec2f74044003'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:09.741685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] The token cache is being invalidate because project 49d86ff517d5432ca8503b690a57c75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:09.742192 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:09.743051 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-19c423f1-0ed1-4778-91e5-ec2f74044003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:09.746021 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1419/2811] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:09 2022] DELETE /identity/v3/projects/49d86ff517d5432ca8503b690a57c75a => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:09.754384 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:09.755067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:09.755067 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:09.791890 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:09.794358 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:09.912738 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1393/2812] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:09.918319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:09.918900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:09.919622 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:09.962781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:09.965960 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:09.987430 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1420/2813] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:09.989012 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:10.004997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed208b9-bcdf-4082-bb17-5f3fc52813ea None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:10.016043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:10.016723 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.017186 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.017380 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:10.017919 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:54:10.018218 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:10.020078 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:10.045429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1394/2814] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:54:10.050530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:10.063224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0cc3e-8ba6-4f13-ba86-c472f20cc064 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:10.072510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:10.073382 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.073518 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.076597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:10.076597 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:10.083249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:10.269354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1421/2815] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:54:10 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:10.275697 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:10.294754 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4b504-a5d3-4d81-a414-f14666cc8ccd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:10.369207 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:10.394297 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:10.395123 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.395423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.395872 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:10.397116 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:10.400245 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:10.706777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1395/2816] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:54:10 2022] POST /identity/v3/users => generated 287 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 16:54:10.711896 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:10.725783 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62952d5d-3934-46af-8309-790ea7540004 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:10.731695 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:10.733978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.733978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.733978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:10.733978 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:10.735120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:10.758013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1422/2817] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:54:10.760523 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:10.780410 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe639b-596c-49a8-9d2b-3f40bf031706 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:10.789559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984e54d-e167-41f1-a496-63bded0a8773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:10.790490 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e54d-e167-41f1-a496-63bded0a8773 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.790981 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e54d-e167-41f1-a496-63bded0a8773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.791406 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984e54d-e167-41f1-a496-63bded0a8773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:10.792704 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984e54d-e167-41f1-a496-63bded0a8773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:10.796398 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984e54d-e167-41f1-a496-63bded0a8773 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:10.811696 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1396/2818] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:54:10.818187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:10.831248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25820f87-e470-4493-ae58-64324b1b5ce8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:10.837767 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:10.838451 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.838733 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.839008 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] PATH_INFO: `/v3/projects/1b37ac6e9f1c4c8da41e43f46f06ebf0/users/0cd24d255c9f47bbaceb0658294c8121/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:10.860722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:10.867122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b37ac6e9f1c4c8da41e43f46f06ebf0, user_id=0cd24d255c9f47bbaceb0658294c8121, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:10.868125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:10.979706 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1423/2819] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:54:10 2022] PUT /identity/v3/projects/1b37ac6e9f1c4c8da41e43f46f06ebf0/users/0cd24d255c9f47bbaceb0658294c8121/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:10.996653 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60f89-b0db-4602-8ef9-48fd5f32b569 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:10.997333 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60f89-b0db-4602-8ef9-48fd5f32b569 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:10.997800 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de60f89-b0db-4602-8ef9-48fd5f32b569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:11.122424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1de60f89-b0db-4602-8ef9-48fd5f32b569 None None] MFA Rules not processed for user `0cd24d255c9f47bbaceb0658294c8121`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:11.124131 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1de60f89-b0db-4602-8ef9-48fd5f32b569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:11.299508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1397/2820] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:11.311723 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:11.315193 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:11.436331 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ae9f4-6882-46c7-81f9-6224d8f631eb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:11.440866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:11.441511 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:11.441793 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:11.442112 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:11.453155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:11.454649 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:11.465983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1424/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:15.460041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:15.460601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:15.460601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:15.485011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] MFA Rules not processed for user `270fe102379a4346ae009a4545865c28`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:15.486129 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:15.571773 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1398/2822] 10.222.0.17 () {58 vars in 967 bytes} [Wed Aug 24 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:15.600380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:15.600380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:15.681677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0dbc-6c68-45f6-bc0b-91bd308f2035 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:15.688755 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f762db26df84f5faf7e6778383e2870', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:15.689835 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:15.690521 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:15.691078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:15.708860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:15.714709 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:15.726332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1425/2823] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Aug 24 16:54:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 16:54:21.702028 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:21.703574 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:21.776642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed02e8-9aac-4f03-af7a-54a40712855f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:21.784083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:21.785041 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:21.785441 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:21.785900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] PATH_INFO: `/v3/users/651140ed5a704cde978be56393e0537e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:21.788561 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=651140ed5a704cde978be56393e0537e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:21.791795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:21.799750 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:22.063654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651140ed5a704cde978be56393e0537e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.195559 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651140ed5a704cde978be56393e0537e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.196236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651140ed5a704cde978be56393e0537e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.211477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651140ed5a704cde978be56393e0537e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.226468 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-23510566-e647-45b3-a998-60f2ee172ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651140ed5a704cde978be56393e0537e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.234787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1399/2824] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:21 2022] DELETE /identity/v3/users/651140ed5a704cde978be56393e0537e => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:22.603023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:22.621530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f840fac-0464-4421-bc83-d26a6ddf48ab service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:22.675070 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:22.675701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:22.675984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:22.676254 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] PATH_INFO: `/v3/projects/19ee69068eb94cf1b4a0d78f7d5e4a8b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:22.678067 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ee69068eb94cf1b4a0d78f7d5e4a8b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:22.679988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:22.791086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:22.808181 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ee69068eb94cf1b4a0d78f7d5e4a8b', 'request_id': 'req-08df955c-0f3d-449f-915a-30819ee5c524'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.814481 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ee69068eb94cf1b4a0d78f7d5e4a8b', 'request_id': 'req-08df955c-0f3d-449f-915a-30819ee5c524'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.929636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ee69068eb94cf1b4a0d78f7d5e4a8b', 'request_id': 'req-08df955c-0f3d-449f-915a-30819ee5c524'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.941064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] The token cache is being invalidate because project 19ee69068eb94cf1b4a0d78f7d5e4a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:22.941358 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.941996 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08df955c-0f3d-449f-915a-30819ee5c524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:22.943607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1426/2825] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:22 2022] DELETE /identity/v3/projects/19ee69068eb94cf1b4a0d78f7d5e4a8b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:22.948752 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:22.949236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:22.949353 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:22.969642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:22.970844 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:23.035053 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1400/2826] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:23.043904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.044187 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.044516 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.065925 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:23.067125 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:23.076820 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1427/2827] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:23.081256 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.091199 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311274e-78a8-46f6-8cb8-d97805423896 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.095818 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.096439 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.096714 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.096983 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.097636 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:54:23.097931 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.100473 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.112401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1401/2828] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:54:23.118407 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.127858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5a72-9074-4528-b2d1-85397e314a5e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.133205 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.133805 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.134083 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.134354 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.134984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.137105 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.255938 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1428/2829] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:54:23 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:23.262270 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.274757 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46141a-c089-40ae-90bd-215f1225aa2d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.336246 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.336886 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.337159 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.337423 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.338094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.339997 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.435841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1402/2830] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:54:23 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:23.440249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.451499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa43e-98d2-4557-ae18-19661460662c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.456690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.457290 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.457616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.457917 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.459275 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.461131 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.471913 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1429/2831] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:54:23.482639 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.488133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:23.494520 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cae11-e07e-4be0-8bf6-2f58ec6ce11b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.499917 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1db08-ba43-41c0-8ff8-3dc84e5189e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.500562 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1db08-ba43-41c0-8ff8-3dc84e5189e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.500841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1db08-ba43-41c0-8ff8-3dc84e5189e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.501108 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1db08-ba43-41c0-8ff8-3dc84e5189e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.501769 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1db08-ba43-41c0-8ff8-3dc84e5189e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.503409 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1db08-ba43-41c0-8ff8-3dc84e5189e9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.515098 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1403/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:54:23.519284 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.533462 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78e1da-8096-444a-9204-32989d8bae71 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.582779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.582779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.582779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.582779 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] PATH_INFO: `/v3/projects/47674e5c5472411ab96582e8f2c86d61/users/67ecea98c57d465ea57bf5792bc6f356/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.583263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=47674e5c5472411ab96582e8f2c86d61, user_id=67ecea98c57d465ea57bf5792bc6f356, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.583263 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.718722 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1430/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:54:23 2022] PUT /identity/v3/projects/47674e5c5472411ab96582e8f2c86d61/users/67ecea98c57d465ea57bf5792bc6f356/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:54:23.730037 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.730405 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.741661 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.761140 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] MFA Rules not processed for user `67ecea98c57d465ea57bf5792bc6f356`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:23.762519 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:23.832951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1404/2834] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:54:23.844425 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:23.846233 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:23.918644 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0714c91-12bc-470a-a593-f8050d68d31b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:23.924247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbc23a-f180-4e72-9ed5-7dcc7e18c7d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:23.925178 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc23a-f180-4e72-9ed5-7dcc7e18c7d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:23.925530 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc23a-f180-4e72-9ed5-7dcc7e18c7d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:23.925942 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc23a-f180-4e72-9ed5-7dcc7e18c7d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:23.939390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbc23a-f180-4e72-9ed5-7dcc7e18c7d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:23.941745 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbc23a-f180-4e72-9ed5-7dcc7e18c7d9 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:23.954983 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1431/2835] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:25.375150 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:25.376655 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:25.459641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2cf6c-4113-47be-b1f0-0a9707e87f8c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:25.464941 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:25.465607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:25.465877 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:25.466144 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] PATH_INFO: `/v3/users/d2018d33c5434a41b2db48b714ab20c0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:25.467286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2018d33c5434a41b2db48b714ab20c0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:25.469303 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:25.814615 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2018d33c5434a41b2db48b714ab20c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:25.920880 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2018d33c5434a41b2db48b714ab20c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:25.921833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2018d33c5434a41b2db48b714ab20c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:25.933842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2018d33c5434a41b2db48b714ab20c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:25.948777 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2018d33c5434a41b2db48b714ab20c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:25.951047 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:25.955637 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1405/2836] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:25 2022] DELETE /identity/v3/users/d2018d33c5434a41b2db48b714ab20c0 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:25.960838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:25.961682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:25.961682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:25.983553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:25.984866 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:26.046830 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:26.061004 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1432/2837] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:26.073120 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:26.074667 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:26.136985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9c9ba-5c62-4dc3-baed-435d2e75976a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:26.140658 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:26.141265 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:26.141565 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:26.141835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:26.152043 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:26.153544 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:26.164682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1406/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:26.662340 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:26.664007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:26.680403 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b1cdf-230d-4421-bcdd-086398f3336a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:26.711502 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:26.712561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:26.712854 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:26.713126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] PATH_INFO: `/v3/projects/07c197e1b21c4020bdd55f84526fceb3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:26.715803 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c197e1b21c4020bdd55f84526fceb3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:26.717776 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:26.866501 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c197e1b21c4020bdd55f84526fceb3', 'request_id': 'req-d3736144-2223-4ef8-b2bb-ab1863e0ea81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:26.873362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c197e1b21c4020bdd55f84526fceb3', 'request_id': 'req-d3736144-2223-4ef8-b2bb-ab1863e0ea81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:26.969634 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c197e1b21c4020bdd55f84526fceb3', 'request_id': 'req-d3736144-2223-4ef8-b2bb-ab1863e0ea81'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:26.983593 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] The token cache is being invalidate because project 07c197e1b21c4020bdd55f84526fceb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:26.983888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:26.996948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d3736144-2223-4ef8-b2bb-ab1863e0ea81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:26.998804 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1433/2839] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:26 2022] DELETE /identity/v3/projects/07c197e1b21c4020bdd55f84526fceb3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:36.403781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=26 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:36.660593 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:36.662058 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:36.676731 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5688a1-bb9c-4187-ae3f-a88d264d42bf service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:36.759550 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:36.760223 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:36.760556 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:36.760822 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] PATH_INFO: `/v3/users/67ecea98c57d465ea57bf5792bc6f356` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:36.762046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ecea98c57d465ea57bf5792bc6f356)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:36.763987 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:36.979718 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:37.041965 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ecea98c57d465ea57bf5792bc6f356'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:37.155641 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ecea98c57d465ea57bf5792bc6f356'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:37.156393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ecea98c57d465ea57bf5792bc6f356'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:37.164904 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ecea98c57d465ea57bf5792bc6f356'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:37.180906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ecea98c57d465ea57bf5792bc6f356'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:37.192033 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1407/2840] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:36 2022] DELETE /identity/v3/users/67ecea98c57d465ea57bf5792bc6f356 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:37.198273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:37.198273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:37.198273 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:37.221250 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:37.222575 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:37.306577 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1434/2841] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:37.317429 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:37.319225 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:37.384136 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c13d6-718c-404d-8363-9994e94c582f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:37.388022 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880fd7d-9558-440a-b642-13f6302be89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:37.388688 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880fd7d-9558-440a-b642-13f6302be89e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:37.392413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880fd7d-9558-440a-b642-13f6302be89e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:37.392413 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880fd7d-9558-440a-b642-13f6302be89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:37.399434 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880fd7d-9558-440a-b642-13f6302be89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:37.401020 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880fd7d-9558-440a-b642-13f6302be89e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:37.412642 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1408/2842] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:37.886819 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:37.888260 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:37.909196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2624c8-735f-4423-ae3e-6a627b051748 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:37.913520 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:37.914282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:37.914823 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:37.914930 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] PATH_INFO: `/v3/projects/47674e5c5472411ab96582e8f2c86d61` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:37.917433 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=47674e5c5472411ab96582e8f2c86d61)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:37.919390 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:38.101269 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47674e5c5472411ab96582e8f2c86d61', 'request_id': 'req-49e8413e-d317-47ac-96f3-639c38f364ba'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:38.108621 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47674e5c5472411ab96582e8f2c86d61', 'request_id': 'req-49e8413e-d317-47ac-96f3-639c38f364ba'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:38.195860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47674e5c5472411ab96582e8f2c86d61', 'request_id': 'req-49e8413e-d317-47ac-96f3-639c38f364ba'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:38.210674 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] The token cache is being invalidate because project 47674e5c5472411ab96582e8f2c86d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:38.211121 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:38.211682 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-49e8413e-d317-47ac-96f3-639c38f364ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:38.213420 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1435/2843] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:37 2022] DELETE /identity/v3/projects/47674e5c5472411ab96582e8f2c86d61 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:38.218643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.218643 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.218942 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.251004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:38.251004 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:38.273350 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:38.324511 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1409/2844] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:38.391863 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.392224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.392743 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.417629 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:38.418858 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:38.428117 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1436/2845] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:38.433988 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:38.446474 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19e951-b1ee-42ba-be8e-e81222dadcf8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:38.451168 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:38.451771 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.452046 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.452307 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.452978 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:54:38.453267 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:38.454875 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:38.467209 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1410/2846] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:54:38.472122 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:38.481815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b6b75-a0a3-4219-b9bc-87ac61d38c18 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:38.486761 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:38.487398 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.487677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.487948 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.488620 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:38.491297 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:38.608882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1437/2847] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:54:38 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:38.613276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:38.635368 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bc061-38c0-4b94-abe3-d5ffc9613882 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:38.698185 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:38.698702 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.699017 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.699282 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.699985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:38.701944 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:38.817972 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1411/2848] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:54:38 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:38.822387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:38.833499 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982a566-8fa3-4f51-9e2d-be87d6f46906 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:38.839157 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:38.839999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.839999 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.840277 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.840988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:38.842614 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:38.853914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1438/2849] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:54:38.858190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:38.870569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27459c69-520d-413f-8103-2edbb7b82799 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:38.875585 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59b5c3-edc2-4626-b2ef-7911e6d4e3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:38.876205 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b5c3-edc2-4626-b2ef-7911e6d4e3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.876516 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b5c3-edc2-4626-b2ef-7911e6d4e3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.876790 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59b5c3-edc2-4626-b2ef-7911e6d4e3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.877446 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59b5c3-edc2-4626-b2ef-7911e6d4e3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:38.879173 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59b5c3-edc2-4626-b2ef-7911e6d4e3f4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:38.891878 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1412/2850] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 16:54:38.897001 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:38.907513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26504abc-3022-4b9c-a6b2-cdaf23b65c78 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:38.914829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:38.914829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:38.914829 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:38.915380 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] PATH_INFO: `/v3/projects/0c6157140b76496d8d6d05bfff3a9121/users/5854a29681034733b989bf8c8bc59842/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:38.935818 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6157140b76496d8d6d05bfff3a9121, user_id=5854a29681034733b989bf8c8bc59842, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:38.938548 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:39.021479 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1439/2851] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:54:38 2022] PUT /identity/v3/projects/0c6157140b76496d8d6d05bfff3a9121/users/5854a29681034733b989bf8c8bc59842/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:39.026555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:39.026837 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:39.027122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:39.047329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:39.048571 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:39.114155 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1413/2852] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:39.124894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:39.126781 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:39.151696 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:39.188947 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d989339-8da4-44b9-9713-cca01dc15b4a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:39.192826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05edba-191a-49ca-a1a3-3fb34d02e29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:39.193643 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05edba-191a-49ca-a1a3-3fb34d02e29a service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:39.193921 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05edba-191a-49ca-a1a3-3fb34d02e29a service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:39.194194 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05edba-191a-49ca-a1a3-3fb34d02e29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:39.204207 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05edba-191a-49ca-a1a3-3fb34d02e29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:39.205666 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05edba-191a-49ca-a1a3-3fb34d02e29a service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:39.215881 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1440/2853] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:39.784679 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:39.786341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:39.802493 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47afbc-cb69-4c27-b5c0-f9ddb3720219 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:39.814181 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:39.815157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:39.815606 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:39.816006 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] PATH_INFO: `/v3/users/0cd24d255c9f47bbaceb0658294c8121` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:39.818455 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd24d255c9f47bbaceb0658294c8121)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:39.822066 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:39.857277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:40.033726 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd24d255c9f47bbaceb0658294c8121'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:40.167656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd24d255c9f47bbaceb0658294c8121'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:40.167656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd24d255c9f47bbaceb0658294c8121'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:40.172847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd24d255c9f47bbaceb0658294c8121'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:40.186827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd24d255c9f47bbaceb0658294c8121'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:40.193393 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1414/2854] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:39 2022] DELETE /identity/v3/users/0cd24d255c9f47bbaceb0658294c8121 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:40.198012 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0267-37ed-47a7-84cb-bbd95cfe9cc9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:40.198478 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0267-37ed-47a7-84cb-bbd95cfe9cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:40.198624 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0267-37ed-47a7-84cb-bbd95cfe9cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:40.218400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7d4f0267-37ed-47a7-84cb-bbd95cfe9cc9 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:40.219616 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7d4f0267-37ed-47a7-84cb-bbd95cfe9cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:40.285335 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1441/2855] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 16:54:40.296211 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:40.298081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:40.369190 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1e88a-a938-4513-b300-9a32716e7be3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:40.372833 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:40.373578 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:40.373867 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:40.374133 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:40.383900 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:40.385391 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:40.395174 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1415/2856] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:40.654591 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75031898-696e-4877-8d9e-132d079d0ced None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:40.654882 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75031898-696e-4877-8d9e-132d079d0ced None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:40.655155 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75031898-696e-4877-8d9e-132d079d0ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:40.715607 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-75031898-696e-4877-8d9e-132d079d0ced None None] MFA Rules not processed for user `5854a29681034733b989bf8c8bc59842`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:40.717726 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-75031898-696e-4877-8d9e-132d079d0ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:40.832904 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1442/2857] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:40.847112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:40.848780 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:40.864790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75031898-696e-4877-8d9e-132d079d0ced None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:40.867877 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-75031898-696e-4877-8d9e-132d079d0ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:40.887734 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75031898-696e-4877-8d9e-132d079d0ced None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:40.893216 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:40.899197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:40.899197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:40.899197 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] PATH_INFO: `/v3/projects/1b37ac6e9f1c4c8da41e43f46f06ebf0` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:40.901362 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b37ac6e9f1c4c8da41e43f46f06ebf0)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:40.903572 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:40.925575 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815762cc-1f7a-4cf0-b531-c40a991e11c4 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:40.931060 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a6b49-a1b6-44cd-9012-3747f088724c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:40.931682 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a6b49-a1b6-44cd-9012-3747f088724c service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:40.931952 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a6b49-a1b6-44cd-9012-3747f088724c service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:40.932612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a6b49-a1b6-44cd-9012-3747f088724c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:40.942216 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a6b49-a1b6-44cd-9012-3747f088724c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:40.944821 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a6b49-a1b6-44cd-9012-3747f088724c service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:40.956897 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1416/2858] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:41.055549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:41.056997 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b37ac6e9f1c4c8da41e43f46f06ebf0', 'request_id': 'req-4539ab51-a118-43ab-b942-94285ef23bee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:41.063677 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b37ac6e9f1c4c8da41e43f46f06ebf0', 'request_id': 'req-4539ab51-a118-43ab-b942-94285ef23bee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:41.173251 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b37ac6e9f1c4c8da41e43f46f06ebf0', 'request_id': 'req-4539ab51-a118-43ab-b942-94285ef23bee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:41.186328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] The token cache is being invalidate because project 1b37ac6e9f1c4c8da41e43f46f06ebf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:41.186639 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:41.188116 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4539ab51-a118-43ab-b942-94285ef23bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:41.189747 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1443/2859] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:40 2022] DELETE /identity/v3/projects/1b37ac6e9f1c4c8da41e43f46f06ebf0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 16:54:41.195059 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.195348 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.195601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.215007 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:41.216134 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:41.260820 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:41.280472 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1417/2860] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:41.289574 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.290102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.290102 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.311072 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:41.312294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:41.320965 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1444/2861] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:41.325385 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:41.336549 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4b9a6-5a68-478a-8946-9dfbec38c22b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:41.341312 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:41.341914 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.342189 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.342456 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.343112 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 16:54:41.343407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:41.345069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:41.357995 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1418/2862] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Aug 24 16:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 16:54:41.362513 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:41.373120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7e45c-dd88-4106-969c-bff3f89c74ce None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:41.378249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:41.378860 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.379138 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.379415 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.380392 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:41.382714 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:41.569311 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1445/2863] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Aug 24 16:54:41 2022] POST /identity/v3/projects => generated 397 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:41.574659 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:41.596286 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327b204-5d02-412d-ad81-3cc9a8dec020 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:41.662430 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:41.663278 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.663553 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.663827 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.664500 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:41.666479 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:41.779296 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1419/2864] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Aug 24 16:54:41 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 16:54:41.783447 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:41.795774 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa0cb8-1357-46ca-b2d9-66465c373b08 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:41.800738 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:41.801338 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.801610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.801991 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.802701 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:41.804288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:41.815064 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1446/2865] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:54:41.819864 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:41.834236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80642f-4835-402f-8978-046dc5713be6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:41.844247 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633292a6-3e06-4715-83de-bf9a2ec825aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:41.844951 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633292a6-3e06-4715-83de-bf9a2ec825aa admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.845341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633292a6-3e06-4715-83de-bf9a2ec825aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.845621 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633292a6-3e06-4715-83de-bf9a2ec825aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.846279 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633292a6-3e06-4715-83de-bf9a2ec825aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:41.848157 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633292a6-3e06-4715-83de-bf9a2ec825aa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:41.860025 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1420/2866] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Aug 24 16:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 16:54:41.865085 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:41.879500 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7710b2-4479-400c-9fa2-431a2f21ea7a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:41.885861 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:41.886538 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:41.886962 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:41.887248 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] PATH_INFO: `/v3/projects/f00e782340194920ad30953f6443d3c8/users/024d307d62804ccd8a3fc988ba925867/roles/af9a72cd6bb0442f9ebb70e3f609fb5c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:41.906495 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00e782340194920ad30953f6443d3c8, user_id=024d307d62804ccd8a3fc988ba925867, role_id=af9a72cd6bb0442f9ebb70e3f609fb5c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:41.909213 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:42.011969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1447/2867] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Aug 24 16:54:41 2022] PUT /identity/v3/projects/f00e782340194920ad30953f6443d3c8/users/024d307d62804ccd8a3fc988ba925867/roles/af9a72cd6bb0442f9ebb70e3f609fb5c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:42.024164 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ad4f-dbb1-473e-8673-38176b183b5f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:42.024477 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ad4f-dbb1-473e-8673-38176b183b5f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:42.024779 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2ad4f-dbb1-473e-8673-38176b183b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:42.057596 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-59b2ad4f-dbb1-473e-8673-38176b183b5f None None] MFA Rules not processed for user `024d307d62804ccd8a3fc988ba925867`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:42.058856 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59b2ad4f-dbb1-473e-8673-38176b183b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:42.127603 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1421/2868] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:42.147469 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:42.149482 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:42.210934 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529b51-66bd-4e5e-a0df-f7b10911326b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:42.214715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:42.215328 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:42.216086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:42.216086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:42.224893 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:42.226321 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:42.236710 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1448/2869] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:42.968630 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:42.969220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:42.969220 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:42.988793 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] MFA Rules not processed for user `5854a29681034733b989bf8c8bc59842`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:42.990139 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:43.032811 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:43.049535 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1422/2870] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:43.088164 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:43.089652 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:43.162698 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af402d7-22b0-4d4a-bcd0-fabb2456ee05 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:43.167529 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43cdc8de558d4087bed8a3bb48cf4bfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:43.169815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:43.169815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:43.169815 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:43.171262 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:43.205336 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:43.260196 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:43.261724 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:43.277300 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1449/2871] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:44.031971 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:44.033612 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:44.037453 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:44.038838 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:44.113615 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a52f1-72b2-4f38-8e1f-aed11d7cf684 service glance] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:44.118635 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f991003-adb8-4a76-afd2-8c8839cd7b35 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:44.120282 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:44.121288 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:44.121750 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:44.122198 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:44.124919 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:44.126143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:44.127100 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:44.127541 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:44.127991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:44.131152 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:44.147299 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:44.148094 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:44.148900 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:44.149730 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:44.168092 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1423/2872] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:44.168329 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1450/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 16:54:49.803943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:49.814547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e6f88-927e-4d69-9b56-e4fd212722d6 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:49.819130 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:49.819741 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:49.820019 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:49.820289 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:49.822114 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:49.832435 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:49.834619 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:49.843011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1424/2874] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:50.651359 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:50.652910 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:50.667957 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe76154-ca5d-441e-8646-e460595fba01 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:50.674219 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:50.674846 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:50.675126 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:50.675400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:50.678547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:50.691208 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:50.693068 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8101bef-a67f-4eea-a59a-188a68ccb465 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:50.704963 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1451/2875] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:51.684170 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:51.772057 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6f4fe-4fa9-4f9d-8513-b4ca9c7e4d95 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:51.776906 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:51.777542 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:51.777815 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:51.778083 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] PATH_INFO: `/v3/users/024d307d62804ccd8a3fc988ba925867` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:51.779242 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=024d307d62804ccd8a3fc988ba925867)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:51.781277 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:52.026310 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024d307d62804ccd8a3fc988ba925867'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:52.129513 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024d307d62804ccd8a3fc988ba925867'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:52.130810 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024d307d62804ccd8a3fc988ba925867'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:52.140947 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024d307d62804ccd8a3fc988ba925867'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:52.144569 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:52.158503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024d307d62804ccd8a3fc988ba925867'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:52.167830 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1425/2876] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:54:51 2022] DELETE /identity/v3/users/024d307d62804ccd8a3fc988ba925867 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:54:52.173623 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:52.174510 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:52.175473 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:52.202007 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] MFA Rules not processed for user `1440567f43e04ab1bb7a5b1e330db669`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:54:52.203732 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:52.285177 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1452/2877] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:54:52.296222 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:52.298023 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:52.365551 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275e88c-f0ad-4c18-8ce6-cca15a18a85f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:52.369323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f494d3b-2419-4884-9388-ab19cfed52fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:52.369943 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f494d3b-2419-4884-9388-ab19cfed52fe service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:52.370217 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f494d3b-2419-4884-9388-ab19cfed52fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:52.370488 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f494d3b-2419-4884-9388-ab19cfed52fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:52.384644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f494d3b-2419-4884-9388-ab19cfed52fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:52.384644 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f494d3b-2419-4884-9388-ab19cfed52fe service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:52.396052 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1426/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:54:52.854146 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:54:52.855610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:54:52.870371 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60198e9e-c5ba-47ab-8dc9-9f709bc3c987 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:54:52.875375 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:54:52.876241 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:54:52.882770 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:54:52.883091 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] PATH_INFO: `/v3/projects/f00e782340194920ad30953f6443d3c8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:54:52.888471 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00e782340194920ad30953f6443d3c8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:54:52.890535 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:54:52.991787 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:54:52.997927 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00e782340194920ad30953f6443d3c8', 'request_id': 'req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:53.004212 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00e782340194920ad30953f6443d3c8', 'request_id': 'req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:53.075180 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00e782340194920ad30953f6443d3c8', 'request_id': 'req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:53.087270 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] The token cache is being invalidate because project f00e782340194920ad30953f6443d3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:54:53.087708 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:53.088797 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-68029d0f-0038-4e19-bc4a-978e5e0ab4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:54:53.090332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1453/2879] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:54:52 2022] DELETE /identity/v3/projects/f00e782340194920ad30953f6443d3c8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:55:02.390656 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=32,keystone:DELETE=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:03.088553 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:38.883678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:38.885309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:38.885309 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:38.910552 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] MFA Rules not processed for user `aaa4ddbee1034946a1f4dfe751a2b5cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:38.912268 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:38.973982 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1427/2880] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:55:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:48.966337 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:55:55.694512 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:55.694915 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:55.695153 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:55.715190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] MFA Rules not processed for user `5854a29681034733b989bf8c8bc59842`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:55:55.716517 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:55.787011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1454/2881] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:55:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:55:56.043467 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:56.045011 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:56.107198 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce212fcf-9d52-45d6-abae-f0e6beef148a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:56.110898 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:56.111503 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:56.111774 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:56.112038 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:56.113847 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:56.124708 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:56.126340 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:56.136772 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1428/2882] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:55:58.086632 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:55:58.088078 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:58.178684 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069d33-4557-4725-bfff-d46ffd2b3128 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:55:58.185711 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:55:58.186322 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:55:58.186610 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:55:58.186887 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:55:58.188832 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:55:58.202821 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:55:58.204470 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:55:58.215972 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1455/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:55:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:03.034486 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:56:03.036069 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:03.057236 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eccdaf-4180-40b9-935d-191d4a0d5d29 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:56:03.065501 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:56:03.067597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:56:03.067597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:56:03.067597 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:56:03.069233 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:56:03.083191 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:56:03.085994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:56:03.099341 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1429/2884] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:56:08.210661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=59 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:56:13.093985 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=32 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:06.251169 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:06.252715 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:06.351844 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8dedc-4b9b-49ba-9460-2a47e43b6c19 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:06.358109 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:06.358712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:06.358984 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:06.359249 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:06.361032 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:06.373006 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:06.374585 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:06.384841 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1456/2885] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:06.489188 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:06.490676 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:06.567578 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df9470-da7b-496d-9a34-444e1e4222b0 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:06.574248 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:06.574835 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:06.575106 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:06.575375 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] PATH_INFO: `/v3/users/5854a29681034733b989bf8c8bc59842` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:06.576781 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5854a29681034733b989bf8c8bc59842)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:06.578795 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:06.805121 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5854a29681034733b989bf8c8bc59842'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:06.893842 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5854a29681034733b989bf8c8bc59842'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:06.894724 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5854a29681034733b989bf8c8bc59842'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:06.920705 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5854a29681034733b989bf8c8bc59842'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:06.922570 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5854a29681034733b989bf8c8bc59842'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:06.929376 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1430/2886] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 16:57:06 2022] DELETE /identity/v3/users/5854a29681034733b989bf8c8bc59842 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:07.319016 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:07.340712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8d40f-aa82-49b3-9960-793483c755ed service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:07.435400 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:07.436061 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:07.436367 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:07.436638 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] PATH_INFO: `/v3/projects/0c6157140b76496d8d6d05bfff3a9121` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:07.437894 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6157140b76496d8d6d05bfff3a9121)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:07.439859 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:07.543023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6157140b76496d8d6d05bfff3a9121', 'request_id': 'req-067338be-0496-42e0-b485-6af30f2755ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:07.550969 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6157140b76496d8d6d05bfff3a9121', 'request_id': 'req-067338be-0496-42e0-b485-6af30f2755ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:07.619712 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6157140b76496d8d6d05bfff3a9121', 'request_id': 'req-067338be-0496-42e0-b485-6af30f2755ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:07.633278 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] The token cache is being invalidate because project 0c6157140b76496d8d6d05bfff3a9121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 16:57:07.633562 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:07.634259 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 16:57:07.636445 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1457/2887] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 16:57:07 2022] DELETE /identity/v3/projects/0c6157140b76496d8d6d05bfff3a9121 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 16:57:16.925654 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:17.629549 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:30.694509 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:30.695926 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:30.771109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43476d-0676-465e-890c-eebfff625a86 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:30.832214 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:30.832866 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:30.833147 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:30.833420 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:30.835339 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:30.850855 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:30.851322 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5d03d-c9ba-41a3-8de7-468ba45a5f33 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:30.862510 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1431/2888] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:40.858122 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=58 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:57:42.414387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:57:42.415811 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:42.479387 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067338be-0496-42e0-b485-6af30f2755ab admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:57:42.483221 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef06aba40382479289b1e9e19395a2be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:57:42.483826 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:42.484099 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:42.484393 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:42.486123 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:42.546527 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:57:42.547989 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:57:42.557452 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1458/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:57:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:57:51.273935 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:57:51.274491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:57:51.274491 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:57:51.295402 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] MFA Rules not processed for user `aaa4ddbee1034946a1f4dfe751a2b5cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 16:57:51.296699 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:57:51.303883 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1432/2890] 10.222.0.17 () {58 vars in 958 bytes} [Wed Aug 24 16:57:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 16:57:52.553243 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=48 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:58:01.288713 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:59:44.469981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:44.471752 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:44.539076 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624a68f-bfd4-4f10-a8e7-0c4b588cc60e service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:44.544084 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:44.544808 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:44.545301 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:44.545576 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:44.547450 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:44.644563 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:44.647281 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:44.668013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1459/2891] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:45.909660 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:45.922013 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398a182-0dae-44eb-b072-d4f3dfc1b8d3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:45.926555 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:45.927169 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:45.927444 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:45.927712 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:45.929547 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:45.945109 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:45.946632 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:45.959678 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1433/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:52.189721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:52.199883 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f9f0d-50f6-4ccb-a407-8386ae8ef9a7 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:52.204432 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:52.205504 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:52.206721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:52.206721 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:52.208292 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:52.219561 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:52.221661 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:52.231690 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1460/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:54.523693 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:54.525171 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:54.539624 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b777ca-f552-44fe-968f-6ce75effceb9 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:54.545262 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4542cf167cc24738b441af6530dead3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:54.545869 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:54.546143 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:54.546407 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:54.548787 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:54.562285 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:54.564802 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:54.575976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1434/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:59:58.351685 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:59:58.353786 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:58.364120 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0b96f-596c-4847-ab2e-40ba91b35c71 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:59:58.368247 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9bde9ceb3c7c46298626580b62e0ec01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:59:58.368888 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:59:58.369160 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:59:58.369444 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:59:58.371190 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:59:58.441174 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:59:58.442626 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:59:58.452636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1461/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 16:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:00:01.827424 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:01.830538 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:01.845634 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36737a2-b8ca-4135-8fc2-91dbbd91ff4a service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:01.851397 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:01.852077 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:01.852537 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:01.852915 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:01.855382 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:01.871329 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:01.873032 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:01.884962 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1435/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 17:00:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:00:05.885086 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:05.888532 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:05.904545 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40034ca0-ccf7-4bc0-afb5-509f5a353761 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:05.910013 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:05.910648 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:05.910929 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:05.911202 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] PATH_INFO: `/v3/users/aaa4ddbee1034946a1f4dfe751a2b5cd` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:05.912333 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa4ddbee1034946a1f4dfe751a2b5cd)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:05.914289 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:05.933466 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:06.148981 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa4ddbee1034946a1f4dfe751a2b5cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.149631 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa4ddbee1034946a1f4dfe751a2b5cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.156546 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa4ddbee1034946a1f4dfe751a2b5cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.226294 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa4ddbee1034946a1f4dfe751a2b5cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.226923 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa4ddbee1034946a1f4dfe751a2b5cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.251636 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1462/2897] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:05 2022] DELETE /identity/v3/users/aaa4ddbee1034946a1f4dfe751a2b5cd => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:06.501167 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:06.512601 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cb57d-c52f-401e-9499-0d1a342b9634 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:06.572808 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:06.573442 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:06.573715 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:06.573992 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] PATH_INFO: `/v3/projects/f18db8d724d4436cb1721859066ac74d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:06.575119 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18db8d724d4436cb1721859066ac74d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:06.577081 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:06.690070 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18db8d724d4436cb1721859066ac74d', 'request_id': 'req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.752254 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18db8d724d4436cb1721859066ac74d', 'request_id': 'req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.759182 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18db8d724d4436cb1721859066ac74d', 'request_id': 'req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.771323 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] The token cache is being invalidate because project f18db8d724d4436cb1721859066ac74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:06.771607 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.772276 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:06.773870 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1436/2898] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:06 2022] DELETE /identity/v3/projects/f18db8d724d4436cb1721859066ac74d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:06.778011 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:06.779498 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:06.793110 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a05d-5707-4616-b74a-c40377fb36d2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:06.851431 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:06.852054 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:06.852332 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:06.852630 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] PATH_INFO: `/v3/users/eeb945354f9d483784dedda9155cdbd4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:06.853790 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb945354f9d483784dedda9155cdbd4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:06.855700 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:07.046579 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb945354f9d483784dedda9155cdbd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.047914 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb945354f9d483784dedda9155cdbd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.056428 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb945354f9d483784dedda9155cdbd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.140492 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb945354f9d483784dedda9155cdbd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.152958 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb945354f9d483784dedda9155cdbd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.159890 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1463/2899] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Aug 24 17:00:06 2022] DELETE /identity/v3/users/eeb945354f9d483784dedda9155cdbd4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:00:07.433991 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:00:07.435504 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:00:07.453921 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fc5a5-4f7f-4f59-a99e-b5648439b554 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:00:07.507646 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:07.518475 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1440567f43e04ab1bb7a5b1e330db669', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '86cc49d474614954959fd1fd2c69fbe7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:00:07.519401 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:00:07.519680 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:00:07.519976 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] PATH_INFO: `/v3/projects/8e0f1d8b8e0340f4b47820ef3de2ca1a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:00:07.521449 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0f1d8b8e0340f4b47820ef3de2ca1a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:00:07.523203 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:00:07.629841 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0f1d8b8e0340f4b47820ef3de2ca1a', 'request_id': 'req-623131eb-cdc2-4104-8072-095b0d7585fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.705852 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0f1d8b8e0340f4b47820ef3de2ca1a', 'request_id': 'req-623131eb-cdc2-4104-8072-095b0d7585fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.713105 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0f1d8b8e0340f4b47820ef3de2ca1a', 'request_id': 'req-623131eb-cdc2-4104-8072-095b0d7585fb'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.724709 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] The token cache is being invalidate because project 8e0f1d8b8e0340f4b47820ef3de2ca1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:00:07.724994 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.725640 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-623131eb-cdc2-4104-8072-095b0d7585fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:00:07.727365 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1437/2900] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Aug 24 17:00:07 2022] DELETE /identity/v3/projects/8e0f1d8b8e0340f4b47820ef3de2ca1a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:00:17.156059 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=19,keystone:SELECT=73,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:00:17.721508 dv-cinder-731167-2-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:02:35.694554 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:02:35.695988 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:35.859319 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7702659-079c-4bbf-9d16-e517df6cd06b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:02:35.921069 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0234ca49360c4a7cbbaec848f27b89ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed3e2826fd1c4c16a6a99d0fc206b22e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:02:35.921679 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:02:35.921954 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:02:35.922224 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:02:35.923945 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:02:35.936547 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:02:35.939515 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf84a38-b877-43c7-b2bc-03e48c66238f service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:02:35.948023 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1464/2901] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Aug 24 17:02:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:02:45.943746 dv-cinder-731167-2-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}